TL;DR: We shipped TestChimp Studio—the companion desktop app that is your IDE for software delivery. Going from “what are we building?” to “is it ready to ship?” no longer means jumping across a myriad of platforms and stitching context by hand. Studio consolidates requirement planning, agentic QA, release governance, issues, executions (E2E, API, performance, security, UX), coverage, and QA knowledge-base exploration in one local workspace. Built on OpenCode: use ChimpHands, bring your own LLM API key, or connect a local model. macOS, Windows, and Linux—or npx @testchimp/studio.

Product page: testchimp.io/studio. Docs: TestChimp Studio · Download.

The stitching tax​

Agentic QA is working. Teams already run /testchimp workflows, keep plans in Git, gate releases on evidence, and let automations start ChimpHands when something changes.

The remaining bottleneck is the workspace. Delivery still looks like this:

Agents compressed doing the work. They did not invent a control pane that sits on your machine next to the repo.

Studio is that pane—in the convenience of a desktop app.

What is TestChimp Studio?​

TestChimp Studio is TestChimp’s companion desktop application. It is not a second product with a second source of truth. It is the same control plane as the web app, plus a local OpenCode agent runtime, so planning and execution share one window.

You can:

• Plan requirements — organize stories and scenarios, then hand them to agents without leaving the workspace (requirement planning).
• Run agentic QA — create tests, upkeep the suite, explore UX, implement a story—catalog workflows with full local checkout context.
• Govern the release — test runs, checks, and readiness on the version you are about to push (release governance).
• Track issues — agent-found and human-found, tied back to plans and executions.
• See executions — E2E, API, performance, security, and UX evidence without a dashboard scavenger hunt.
• Govern coverage — requirements, API schemas, real-user behaviour (TrueCoverage), and the gaps you should close next.
• Explore the QA knowledge-base — stories, scenarios, tests, releases, issues, and how they relate (QA Brain / InfiniTrace).

The desktop binary is the shell + agent. The product UI still loads from your App URL and ships independently—so Studio does not freeze the product on whatever build you installed last month.

Full walkthrough: TestChimp Studio.

One workspace, the whole delivery loop​

If you have been living in the TestChimp loop, the pieces already exist as separate surfaces. Studio is the place they stop competing for focus.

Requirement (story / scenario)
        ↓
Agentic workflow  (/testchimp create-tests, run QA, upkeep, …)
        ↓
Executions  (SmartTests, API, k6, ExploreChimp, security)
        ↓
Coverage + issues  (plans, schemas, RUM, QA Brain)
        ↓
Release  (evidence, checks, ship / don’t ship)

QA can run autonomously—local agent now, cloud ChimpHands and automations when you want CI to do it overnight—while you keep complete transparency into posture and how it aligns with product intent.

That is the difference between “we have an agent” and “we have a delivery workspace.”

Your agent, your keys, your machine​

Studio is built on OpenCode. Settings → LLM is explicit about who pays for tokens:

Studio starts one OpenCode serve for the active workspace and keeps the TestChimp skill current. You are not pasting mcp.json into a third IDE to get the catalog. You also are not required to burn a Cursor or Claude seat for every /testchimp upkeep you run at the desk.

Cloud ChimpHands stays the right executor for async CI. Studio is the right executor for interactive delivery on the laptop that already has the branch.

Why this matters in the agentic era​

We have argued that skills are SaaS distribution—the playbook travels with the agent. That automations close the routing gap. That requirements need governance before agents spend tokens. That release governance is the ship decision, not a spreadsheet.

Studio closes the attention gap.

Without it, the platform is still a tab, the agent is still another tab, and the repo is still a third window. With it, the control pane and the agent share a desktop—requirement planning all the way to a confident release push.

That is how you boil the QA lake without boiling your context-switching budget.

Frequently asked questions​

What is TestChimp Studio?​

TestChimp Studio is TestChimp’s companion desktop app—an IDE for software delivery. It consolidates requirement planning, agentic QA workflows, release governance, issues, test executions, coverage insights, and QA knowledge-base exploration in one workspace on your machine.

How do I download TestChimp Studio?​

Use the official Download TestChimp Studio page. Certified installers are on GitHub Releases for macOS, Windows, and Linux. Cross-platform: npx @testchimp/studio.

Does Studio replace the TestChimp web app?​

No. Studio loads the same product UI from your configured App URL. The web app remains the shared control plane and the URL you send a teammate. Studio adds the local shell and agent runtime.

How is Studio different from Cursor or Claude Code?​

Those IDEs are excellent for implementation. Studio is a delivery workspace: plans, releases, executions, coverage, and catalog /testchimp workflows sit next to the local agent. You can still use Cursor/Claude for feature work; Studio is where QA posture and product intent stay visible.

Can I use my own LLM or a local model?​

Yes. Settings → LLM supports ChimpHands, BYO API keys (OpenAI, Anthropic, ChatGPT OAuth, models.dev providers), and a custom base URL for local or self-hosted models. BYO and custom paths do not consume ChimpHands credits.

Is TestChimp Studio available on macOS, Windows, and Linux?​

Yes. Certified installers ship for those platforms, and npx @testchimp/studio runs on all three. See Download.

Do I need to re-map my project if I switch between npx and the installer?​

No. Certified builds and npm paths share ~/.testchimp and the same Electron userData.

Try it​

1. Open Download TestChimp Studio
2. Install via your platform’s certified build—or run npx @testchimp/studio
3. Sign in and map a local project folder
4. Open a story → create tests, or open Releases and look at the version you are about to ship
5. Set Settings → LLM to ChimpHands, BYO, or local

Start here:

• TestChimp Studio overview
• Download & install
• Studio updates
• Workflows
• ChimpHands
• Release management
• testchimp.io/studio

Your control pane for software delivery—now in the convenience of a desktop app.

Further reading​

TestChimp

• TestChimp Studio
• Download TestChimp Studio
• What is TestChimp?
• Workflows
• ChimpHands
• Requirement planning
• Release management
• TrueCoverage
• InfiniTrace

Related posts

• Meet ChimpHands
• Put QA on Auto-Pilot with Workflow Automations
• Release governance
• Test planning as code
• SKILLs as SaaS distribution
• Boiling the QA lake

Concepts

• OpenCode
• TestChimp skill
• Agent Skills

TL;DR: We shipped ChimpHands—TestChimp's native QA cloud agent. It runs on your GitHub Actions runner (full code checkout, your secrets, your branches), executes the same /testchimp workflows you already use locally, and stays wired into the platform for automations, workflow executions, and live chat. Skill + CLI are packaged on every run—no mcp.json wiring per host. Attach stories, scenarios, issues, releases, and batch invocations from chat; add repo line ranges, upload local files, and co-edit in the diff pane. Built on OpenCode with GPT 5.6-range models tuned for cost-effective QA—not your Cursor or Claude dev subscriptions. The differentiator: async runs and interactive clarification share one session. Fire an automation overnight; jump into the same ChimpHands chat when the agent needs you. One-click setup.

Full docs: ChimpHands.

The wrong agent for the job (and the wrong bill)​

Agentic QA is working. Teams run /testchimp fix-issue, /testchimp upkeep, /testchimp author-plans—and Automations route those workflows when issues, batches, and releases change.

But the executor still matters:

We built ChimpHands to be the native answer: QA workloads on your CI, orchestrated by TestChimp, metered for QA, traceable at the workflow level.

What is ChimpHands?​

ChimpHands is TestChimp's own cloud agent for QA—not a generic coding-agent integration you bolt on later.

• Runs on your repo's GitHub Actions runner — not on TestChimp VMs. The agent checks out your branch, uses your tree, and opens PRs from CI.
• Full code context — same environment your SmartTests and policies already live in.
• Same workflow catalog — fix-issue, implement, upkeep, author-plans, instrument-truecoverage, run-qa, and the rest. One skill, one MCP, one workflow-execution-id timeline.
• OpenCode + TestChimp skill — each job installs OpenCode and @testchimp/cli, clones testchimp-skills, and runs testchimp chimphands serve.
• Cost-optimized models — LLM traffic routes through GPT 5.6-range models suited to playbook-driven tool use, not always the priciest frontier tier.

Think: the QA agent your platform already knows how to schedule, audit, and budget.

Platform context and IDE-agent ergonomics—without the wiring tax​

BYO cloud agents mean committing mcp.json, configuring MCP in Cursor/Copilot dashboards, and pasting issue ids into prompts. ChimpHands ships differently:

Packaged on CI: every job installs OpenCode, @testchimp/cli@latest, and clones testchimp-skills. You set TESTCHIMP_API_KEY once as a GitHub secret—not per-agent MCP setup.

Platform entities in chat: the composer + menu attaches removable tags for stories, scenarios, issues, tests, releases, and automation batches—searchable pickers, stable ref: lines in the outbound prompt (ordinal ids where applicable). Handoffs from Plans and Issues can pre-fill that context.

Repo + file parity with typical agents:

So you get the "attach the failing test batch and lines 42–88 of checkout.spec.ts" workflow—native to TestChimp, not duct-taped through a generic coding-agent product.

Details: Chat with platform and repo context.

How it runs (one-click, then forget)​

Setup is deliberately boring—in a good way:

1. Connect the TestChimp GitHub App and map your repository.
2. Open ChimpHands in the sidebar → install .github/workflows/chimphands.yml (direct commit or PR if branch rules require it).
3. Add TESTCHIMP_API_KEY as a GitHub Actions secret.

After that, ChimpHands sessions start from chat, Plans (Spec out / Scope out / Implement / Test), Issues, or Automations when you pick ChimpHands as the invocation strategy.

Trigger (automation / chat / Plans handoff)
        ↓
ChimpHands session + workflow execution
        ↓
GitHub Actions: checkout → OpenCode → TestChimp skill → /testchimp …
        ↓
Tunnel ↔ TestChimp UI (stream tools, diffs, messages)
        ↓
Pull request + Done / Failed in Workflow Executions

You can always drop to the GitHub Actions run for raw logs. The product UI is for humans; CI is the source of truth for the run.

Mechanics: How ChimpHands runs.

Async → in-convo without losing the thread​

Most cloud agents force a tradeoff:

• Async: great for overnight upkeep, terrible when the agent hits an ambiguous requirement at 2am.
• Interactive: great for pairing, terrible for "fix every failed batch without me watching."

ChimpHands bridges both on one session id:

External agents (labelled issues, separate cloud UIs) usually break continuity here—you re-paste context or start over. ChimpHands keeps one session, one workflow execution, one Actions run.

That's the seamless human-in-the-loop story we wanted for policy-traceable workflows: attend when it matters, ignore when it doesn't—without losing the audit trail.

Why separate QA agent economics​

ChimpHands metered usage enables you to have your QA workloads isolated from dev-agent subscriptions.

Your Claude Code and Cursor seats stay for implementation. ChimpHands credits cover the long tail:

• Fix failing SmartTests after CI
• Windowed upkeep digests
• author-plans with full repo context
• TrueCoverage instrumentation and gap fixes
• Release run-qa composites

You set a QA budget per billing cycle. The platform enforces gates before metered runs and shows remaining credits in-product. Within that budget, ChimpHands aims for the best feasible protection—high-signal automations first, predictable spend, fail-closed when credits run out.

Value prop deep dive: Why ChimpHands.

Automations, now native​

When we shipped Workflow Automations, the missing piece was a first-class executor that didn't require you to operate another agent product.

ChimpHands is now the recommended invocation strategy:

Configure once under Workflows → Automations, choose ChimpHands, tune aggregation and gates. Same workflow-execution-id reporting as local /testchimp.

Recipes: Typical automation setups · Best practices.

ChimpHands vs bring-your-own cloud agents​

Bring-your-own paths remain: Cloud agents · Setting up cloud agents.

Why this matters in the agentic era​

We've argued that skills are SaaS distribution—the playbook travels with the agent. That automations close the routing gap—events in, workflow executions out. That requirements need governance before agents spend tokens.

ChimpHands closes the executor gap: a cloud agent that is of TestChimp, on your CI, for QA economics.

Local Cursor / Claude for the interactive dev loop. ChimpHands for the overnight fix, the failed batch, the ready story, the release gate—and for jumping into the same session when the agent has a question.

That's how you boil the QA lake without standing next to every kettle or duct-taping five agent products together.

Frequently asked questions​

What is ChimpHands?​

ChimpHands is TestChimp's native QA cloud agent. It runs catalog /testchimp workflows on your GitHub Actions runner with full repository context, OpenCode, and the TestChimp skill—reachable from automations, Plans, Issues, and in-product chat.

Where does ChimpHands run?​

On your GitHub Actions infrastructure. TestChimp dispatches .github/workflows/chimphands.yml; the job checks out your branch and executes the workflow there. TestChimp orchestrates sessions and streams UI updates over a tunnel—it does not host your source code.

How is ChimpHands different from Cursor or Claude for QA?​

ChimpHands isolates QA spend from dev-agent subscriptions, uses cost-optimized GPT 5.6-range models for playbook-driven work, and ties every run to workflow executions in TestChimp. Use Cursor/Claude for IDE pair-programming; use ChimpHands for async QA, automations, and repo-context plan authoring.

How does human-in-the-loop work?​

Automations can run fully async. If the agent needs clarification, open the same ChimpHands session in TestChimp chat—your reply reaches the CI runtime and the turn continues without a new conversation or copy-pasted context.

How do I set up ChimpHands?​

Connect the TestChimp GitHub App, map the repo, install the ChimpHands workflow from the ChimpHands page, and add TESTCHIMP_API_KEY as a GitHub Actions secret. Step-by-step: ChimpHands intro.

Does ChimpHands replace local /testchimp?​

No. Local agents with the TestChimp skill remain ideal for fast iteration and heavy IDE work. ChimpHands is the native cloud path—especially for automations and fire-and-forget QA runs.

Can I still use GitHub Issue or Webhook automations?​

Yes. ChimpHands is the recommended default. Via GitHub Issue and Webhook stay available for bring-your-own agent stacks.

Do I need to commit mcp.json for ChimpHands?​

No. The Actions job installs the TestChimp skill and @testchimp/cli on every run. You only need TESTCHIMP_API_KEY as a GitHub secret—unlike BYO cloud agents that require per-host MCP configuration.

Can I attach platform entities and file ranges in chat?​

Yes. Tag stories, scenarios, issues, tests, releases, and automation batches from the composer + menu; add repo line ranges or whole files from the diff pane or explorer; upload local files as artifacts. Edit files collaboratively in the side pane when the runtime allows.

Try it​

1. Open ChimpHands in your TestChimp project sidebar
2. Connect GitHub and Install the workflow (direct or via PR)
3. Add TESTCHIMP_API_KEY to GitHub Actions secrets
4. Start a chat session—or create an automation with ChimpHands as invocation strategy
5. Watch Executions → Workflow Executions (and join chat if the agent asks)

Start here:

• ChimpHands overview
• Why ChimpHands
• Best practices
• Automations
• Workflows catalog

When QA work needs doing, the agent should already be on your CI—not eating your dev subscription or waiting for you to wire another product.

Further reading​

TestChimp

• ChimpHands
• Why ChimpHands
• ChimpHands best practices
• Automations
• Cloud agents
• Fix issue
• Upkeep

Related posts

• Put QA on Auto-Pilot with Workflow Automations
• Policy-traceable workflows
• SKILLs as SaaS distribution
• Test planning as code
• Boiling the QA lake

Concepts

• OpenCode
• TestChimp skill
• Agent Skills

TL;DR: Agents built the product. Agents authored the tests that verify the implementation. The missing piece is accountability—has anyone actually looked at the test to ensure it truly verifies the scenario? Verified Tests puts that sanity check in the platform: inspect screen captures, steps, and code, then stamp a Verified Badge on the covering scenario.

Full product guide: Verified Tests.

The missing piece is accountability​

Agents are very good at getting a path green.

The SmartTest runs. The scenario is linked. Requirement coverage looks healthier than last sprint.

That is not the same as: this test actually verifies the behaviour the scenario describes.

Has anyone actually looked at the test?

A scenario annotation is cheap to add—and agents add them at speed. They can link the right scenario to a script that asserts a toast, skips the important check, or walks a cousin flow that happens to share a button. The dashboard still counts it as coverage.

We used to paper over that with implicit knowledge: the person who wrote the test was the person who knew what it proved. Agents broke that assumption. Volume went up. Inspection did not.

Green is an execution fact. Linked is a mapping. Verified is the accountability layer: a person has sanity-checked that the covering test truly verifies the scenario.

That is the problem Verified Tests solves.

Why this is necessary in the agentic era​

Before agents, test authoring was slow enough that “who wrote this?” was a reasonable proxy for “who checked this?”

That proxy is dead.

Agents implement. Agents author SmartTests. Agents attach scenario ids because the plan files told them to. You still need a durable record that a person opened the execution, looked at what actually happened, read the script, and agreed: yes, this test verifies that scenario.

Otherwise requirement traceability is a spreadsheet of claims with nicer UI. Coverage insights tell you what ran. They do not tell you whether the test is the right test.

Introducing Verified Tests​

TestChimp now lets you Verify tests in the platform—without bouncing out to a repo or a recording elsewhere.

Open a SmartTest execution and you already have what you need to make an informed decision:

• Screen captures from the run
• Steps the test actually took
• The code, via View Test, so you can read the script next to the evidence

Then use the check badge next to the SmartTest name. Same visual language as a verify badge elsewhere on the internet, because the job is the same: this was inspected.

Three states on the test:

• Hollow — none of the linked scenarios are verified
• Grey — some, not all
• Blue — every linked scenario for that test is verified

For one scenario, you confirm. For several, you verify per scenario—or Mark all as verified. Hover a filled badge and you see who verified it.

Un-verify is deliberate: it records manually unverified, not “never happened.” The audit trail stays.

Status lives in TestChimp, not in source. Agents can keep authoring Playwright. Humans stamp the claim when they have actually looked.

A Verified Badge on scenario coverage​

Plans, requirement coverage, and test runs already show recent execution results per scenario.

You now also see a Verified Badge on each scenario—an extra layer of assurance that the product is being tested properly, not only that something ran green.

• Filled — a user has sanity-checked a covering test for that scenario
• Hollow — the scenario has tests, but nobody has inspected them yet
• Omitted — no linked tests, so there is nothing to verify

Clicking the coverage-side badge does not flip the bit. It tells you how: open an execution, inspect the run, then use the check badge next to the test name. Verification is an inspection action, not a bulk paint on a dashboard.

That is the point. If it were too easy, we would have rebuilt the original lie at a larger scale.

How this fits the rest of TestChimp​

If you’ve been following along:

• Test Planning as Code puts stories and scenarios in Git so agents have a real task
• Scenario annotations declare which SmartTest claims which scenario
• Requirement traceability rolls execution results up the plan
• Verified Tests records that a human checked the claim—not just that CI was green

Planned reality → linked automation → inspected coverage → release confidence. The middle of that chain was the hole once agents started writing both sides.

Frequently asked questions​

What is a verified test?​

A verified test in TestChimp is a SmartTest a project member has inspected and confirmed actually covers a linked scenario. It is more than a green run or a scenario annotation: someone looked at the screen captures, steps, and code, and stamped the claim.

How is verified different from linking a test to a scenario?​

Linking (a Playwright annotation with type: 'scenario' and #TS-<n>) is a coverage claim. Verification is a human sanity check of that claim. You can have linked, green, and still unverified.

Where do I verify a SmartTest?​

Open a SmartTest execution (click a coverage square). Inspect the screen captures, steps, and test code, then use the check badge after the test name. Coverage-train badges are read-only hints.

What does the Verified Badge on a scenario mean?​

A filled badge means at least one covering SmartTest has been manually verified for that scenario. Hollow means tests exist but have not been sanity-checked yet. No linked tests → no badge.

Does renaming a scenario reset verification?​

No, as long as the scenario ordinal (#TS-n) stays the same. Linking a different scenario is a new claim and starts unverified.

Can I un-verify?​

Yes. Confirming un-verify sets the link to manually unverified (distinct from never verified). Who/when hover applies to the verified state.

What if the test has no linked scenarios?​

It cannot be marked verified. There is no scenario claim to inspect. Link the scenario first.

When agents run your SDLC, trust—but verify.

Docs: Verified Tests · Link tests to scenarios · Requirement traceability.

Agent coding is very good at getting the feature to work.

A user can check out. The API returns 200. The SmartTest is green. You merge.

Then ten concurrent checkouts queue behind one chatty query. Or the reports page, which was snappy with three invoices in the seed DB, falls over on a tenant that actually uses the product.

That gap is not a mystery. Agents optimize for the path in front of them—one user, empty-ish data, mocked collaborators that return in 0 ms. Concurrency and data volume are different questions. Functional tests do not answer them.

So we shipped performance testing as a first-class TestChimp surface: k6 in your repo, agentic workflows that author the right journeys (and keep composites honest), related runs after a PR or a release, and an Executions view that compares this run to a prior one.

Full product guide: Performance Testing.

The questions that actually matter​

Not “did k6 print a chart.” The questions product and eng already have:

We keep load and volume as separate axes. “Make it heavier” by turning both knobs hides which one broke.

Agents author journeys. You still own capacity.​

/testchimp create-perf-tests does not invent a k6 file from vibes.

It ranks real scenarios (priority, semantic coverage, get-requirement-coverage --include-perf). It uses redacted REAL E2E interaction shapes—method, path template, schema, status class, timing distribution—not cookies, tokens, or production bodies. When TrueCoverage is mature, relative demand helps order the queue and suggest composite weights.

What it will not do: copy a production RPS into load.js. TestChimp telemetry tells you what is worth testing. You (or run-perf-tests.policy.md) still set VUs, duration, and dataset size. Smoke is the default while authoring. Load/volume wait on an explicit capacity decision.

Composites are a weighted mix of journeys—“typical overall load,” not isolated degrade detection. Adding a journey to a composite is always a prompted approval. Silent membership is how you accidentally change the mix and then argue about the chart.

Outbound deps (payments, email, LLMs, partner APIs) get harness mocks with realistic latency. Hitting Stripe in a load test is expensive and flaky. Stubbing it at 0 ms is worse: you never see pool exhaustion. Either outcome is false confidence.

Folder layout, metadata, and wrappers: How performance tests are organized.

Related runs after a PR. Release range when you cut.​

Perf load/volume does not execute inside /testchimp test. Functional smoke should stay fast. k6 execution is /testchimp run-perf-tests or CI k6/scripts/run.sh. Run QA does write plans/smart-smoke/<branch>/related-perf-tests.json when k6/journeys exists, so the next CI --impacted run picks up the affected journeys.

On a PR, we select related journeys from the change set—scenarios, operations, path templates—then run them through k6/scripts/run.sh --impacted. Bare k6 run is not a TestChimp run: no ingest, no Executions charts.

On a release, the prompt from the release page is:

/testchimp run performance tests for release 1.2.0

The git range is prior SHA → cut SHA, not whatever happens to be checked out. Ingest stamps TESTCHIMP_RELEASE so the release panel and Executions list light up. If existing journeys do not cover that range, the agent asks whether to author (nested create-perf-tests on the same range) instead of silently inventing a suite or pretending coverage exists.

/testchimp upkeep-perf is the long loop: stale scenario links, drifted contracts, composite membership, thresholds that should not be quietly weakened to make compare green.

/testchimp init-perf scaffolds k6/ once. /testchimp import-perf-tests brings Locust/JMeter/Gatling/Artillery/k6-from-elsewhere into that tree. Source VU counts stay out of load profiles until you approve capacity.

Workflow walkthrough: Performance testing workflows.

Results you can overlay, not a one-off HTML report​

Wrapper runs ingest a summary and attach downsampled timeseries (p95, fail rate, VUs, and the rest of the k6 dump). Executions → Performance Tests lists them—optionally grouped by batch. Open a run for threshold, p95, fail rate, duration, metric-over-time, and VUs.

Then click Add Comparison Run. Recommended candidates share the comparison keys (environment, profile, dataset, LLM mode, mock/latency profile). The overlay re-bases both series on elapsed time so you are not comparing Tuesday 4pm wall clock to Wednesday 9am.

That is the degrade question: same kind of run, previous comparable result, is this worse? Green against a loose threshold can still be a regression versus last Tuesday. CLI compare-perf-to-baseline is the same contract for agents and CI—it exits nonzero when comparison.regressed is true. A mismatched baseline is incomparable, not a pass.

Viewing results.

How this fits the rest of TestChimp​

If you’ve been following along:

• SmartTests (now grouped as Functional Testing in the docs) prove the path works for a user
• Smart Smoke keeps that suite runnable in a CI budget
• TrueCoverage says which journeys real people actually take
• Performance testing asks whether those journeys still hold when many people take them—or when the tenant already has history

Planned reality → functionally tested reality → load/volume-tested reality → production reality. The middle was the hole for teams whose agents ship faster than anyone can stare at an EXPLAIN plan.

Frequently asked questions​

What is TestChimp performance testing?​

Grafana k6 scripts in your mapped tests folder (k6/journeys and k6/composites), authored and selected by agentic workflows, ingested into Executions so you can chart a run and overlay a comparable prior run.

Do I need this if my SmartTests are green?​

Yes if you care about concurrency or tenant data volume. Agents (and humans) routinely ship designs that are correct for one user on a small seed database and fail under the conditions production actually creates.

Does /testchimp test run k6?​

No. Keep functional PR smoke fast. Use /testchimp run-perf-tests (or a dedicated CI job that calls the same wrappers).

Will TestChimp pick my VU count from production traffic?​

No. TrueCoverage and interaction timings are relative signals—what to test, how to weight a mix. Absolute VUs, RPS, duration, and dataset size come from policy or an explicit approval.

How do I see if we got slower?​

Open the run → Add Comparison Run → pick a recommended candidate. Or compare-perf-to-baseline in CI. Compare only matching environment / profile / dataset / mock profile.

Docs: Intro · Organization · Workflows · Results.

Concept: What is load testing? · Volume testing · k6 vs JMeter vs Locust · Compare across releases.

Smoke testing is a quick, shallow check that a new build works at all (it starts, logs in, and its critical paths respond), while regression testing re-runs a broad set of tests to confirm that existing behaviour did not break after a change. Smoke runs on every build in minutes to decide whether deeper testing is worth starting; regression runs before a release, or nightly, and can take hours.

The trouble starts when the regression suite grows faster than your CI budget. That is the problem Smart Smoke solves.

TL;DR: We’ve shipped API contract governance in TestChimp. Point us at your OpenAPI roots, opt in to API payload capture on SmartTest runs, and the APIs surface shows which operations—and which request/response fields and status codes—your automation actually exercised. Uncovered holes become Create issue, Create test, or Ignore gap—accountable work, not a spreadsheet of “we should cover that someday.”

The contract nobody audits​

Your OpenAPI (or Swagger) file is not documentation fluff. It is the business contract of the product: methods, paths, request shapes, response codes, and fields clients depend on.

UI E2E already touches that contract. Checkout hits POST /orders. Declined payment hits POST /payments. A rare enum or a 422 body? Often never.

Without a contract lens, teams hit the same failure modes:

API contract governance is not a separate API-testing product. It is the audit and triage layer that turns traffic your SmartTests already generate into accountable coverage of the contract.

Full product guide: API Contract Governance.

How it works​

Three inputs. One surface.

1. Discover the contract — Configure OpenAPI root path(s) from your connected repo (APIs → Configure). TestChimp parses them into services and operations.
2. Observe traffic — Run SmartTests with TESTCHIMP_ENABLE_API_CAPTURE=1 (and @testchimp/playwright ≥ 0.2.17). Matching HTTP interactions upload with the batch; coverage denorm runs after—not on the hot ingest path.
3. Govern gaps — List view for the portfolio; detail view for schema fields and response codes. From any hole: file an issue, copy an agent prompt to create/update tests, or ignore the gap when it is intentional.

Capture is off by default so ordinary runs stay light. Opt in when you want Operations coverage—same pattern as other optional runtime features. Setup detail: Set up API contract governance and the runtime plugin.

From endpoint scores to schema fields​

The APIs list is the portfolio view: service picker, coverage-band summary, method/path, spec-defined vs observed-only, coverage score, latency, covering-test chips.

Click a row and you get the interesting part—field-level truth:

• Request schema and query parameters with covering tests per field
• Response schemas nested by status code
• undocumented tags when traffic (or denorm) sees a field or code the OpenAPI root never declared
• Empty covering tests when the contract exists but automation never exercised it

That is the difference between “we hit /orders” and “we never sent delivery.type = same-day, and nobody owns that hole.”

List and detail walkthroughs: Operations list · Operation detail.

Governance that sticks: issue, test, or ignore​

Coverage without triage is another dashboard nobody opens.

Wherever coverage is missing—operation, field, or response code—TestChimp offers the same decision set:

Agents already understand operation-id / field / response-code scopes. The UI just stops the gap from living only in someone’s head.

How this fits the rest of TestChimp​

If you’ve followed us:

• Requirement quality governance hardens the plan contract before agents spend tokens
• Release governance hardens the ship decision for a version
• API contract governance hardens what your automation proved against the HTTP contract—the surface clients and integrations actually depend on

SmartTests remain the traffic source. Issues remain the backlog. Create-tests remains the authoring path. The APIs page is where contract debt becomes visible and governable—next to TrueCoverage for behaviour-aligned RUM, not instead of it.

Planned reality → tested reality → production reality. The API surface was the missing middle for teams whose “coverage” stopped at green UI journeys.

Frequently asked questions​

What is API contract governance in TestChimp?​

API contract governance maps OpenAPI operations to real and mocked SmartTest traffic so you can see endpoint and schema-field coverage, then triage gaps with create issue, create test, or ignore gap.

Do I need dedicated API tests?​

No. Coverage is inferred from HTTP traffic your SmartTests already generate when capture is enabled. You can still author dedicated API specs; the governance surface tracks both.

Why is my coverage empty after configuring OpenAPI?​

Parsing the spec discovers operations. Scores and covering tests need runs with TESTCHIMP_ENABLE_API_CAPTURE=1. Capture is off by default on purpose.

What does Ignore gap do?​

It marks an uncovered operation, field, or response code as intentionally ungoverned for now—without creating an issue or a test—so teams can acknowledge debt without false urgency.

Spec defined No vs undocumented—what’s the difference?​

Spec defined No on an operation means the whole method/path is missing from the OpenAPI root (observed in traffic only). An undocumented tag on a field or response code means that leaf was seen but not declared—classic contract drift.

Try it​

1. Connect git and open APIs → Configure — select your OpenAPI root file(s)
2. Wait for parse/sync; pick the service in the list
3. Run SmartTests with TESTCHIMP_ENABLE_API_CAPTURE=1
4. Refresh APIs — open a low-score or uncovered operation
5. Create issue, Create test, or Ignore gap on the holes that matter

Start here:

• API Contract Governance
• Setup
• List view
• Detail view
• Playwright runtime plugin
• TestChimp skill

Know the contract. Cover the edge cases—before unaudited branches ship.

Further reading​

TestChimp

• API Contract Governance
• Set up API contract governance
• SmartTests runtime plugin
• Create tests
• Issue management

Related posts

• Release Governance: Ship When the Evidence Says So
• Requirement quality governance (DeFOSPAM)
• TrueCoverage
• Boiling the QA lake

TL;DR: Existing QA tooling lets you follow explicit links only—story → ticket → test case → run, if someone remembered to wire them. Anything related by meaning but not linked stays invisible. TestChimp consolidates stories, scenarios, SmartTests, issues, and TrueCoverage events into a single embedding space. Semantic Canvas (in QA Brain) projects that space onto a 2D map so you can navigate neighborhoods, identify clusters, and surface related entities by conceptual similarity—then link near-misses, mark lookalikes distinct, or raise cleanup work from the sidebar.

Explicit links aren’t enough​

Last week we shipped InfiniTrace—walk the structural edges of your QA graph board by board.

That answers:

“What is linked to this story / test / event?”

Most of the industry stops there. Jira, TestRail, CI folders, coverage tools: if the edge wasn’t created, the relationship doesn’t exist. Keyword search helps only when titles share tokens. Paraphrases, cross-type near-misses, and quiet duplicates fall through the cracks—especially when agents author plans and tests at speed.

The sibling question traditional tools never answer:

“What looks like this—even if nobody linked it yet?”

Or:

“Which entities belong to the same conceptual neighborhood?”

Those need meaning, not another list sorted by updated-at.

TestChimp already maps every QA entity into a shared embedding space—beyond explicit links, into semantic proximity. Semantic Canvas is the surface that lets you see and navigate that space, next to InfiniTrace in QA Brain.

Introducing Semantic Canvas​

Semantic Canvas is a spatial map of your project’s QA knowledge—laid out by conceptual similarity.

Open QA Brain → Semantic Canvas tab (/brain?tab=semantic-canvas).

1. Add the entity types you care about—Stories, Scenarios, Tests, Issues, Events
2. Nodes land on a UMAP canvas: high-dimensional embeddings projected so local neighborhoods reflect cosine similarity
3. Pan, zoom, filter, search (#US-12, titles, …)
4. Click a node → the detail sidebar fills with cosine-ranked peers
5. Link, mark distinct, create a duplicate-cleanup issue, or open details—in place

Deep links encode types, filters, border modes, and selection in ?locator= (with tab=semantic-canvas). “Look at this cluster” is a URL, not a screenshot.

Docs: Semantic Canvas.

What you can actually do with it​

A few use cases that show why a meaning map beats link-chasing alone:

Plan neighborhoods with live automation health​

Load Stories + Scenarios + Tests, paint test borders with latest execution status, and scan the map: requirement clusters with failing automation sitting right next to the plans they should cover—without walking each story’s link tree.

Duplicate / lookalike issue clusters​

Load Issues alone with severity borders. Twin bugs filed weeks apart with different titles collapse into neighborhoods. Open the sidebar, mark distinct where needed, or file cleanup when it’s a true duplicate.

Close semantic coverage gaps​

Put Scenarios + Tests (or Events + Tests) on one canvas. Select an unlinked scenario or hot TrueCoverage event → Close by tests ranks peers by cosine similarity → Link the near-miss that should have been wired months ago.

Requirement overlap before you plan more​

Plot Stories + Scenarios. Dense blobs often mean one idea wearing three titles—or a story that needs a scenario split—before you (or an agent) author more surface area.

TrueCoverage hotspots vs automation islands​

Add Events with has test coverage borders. Uncovered events that sit next to well-covered automation clusters are high-ROI gaps: instrumentation, missing emits, or a test that almost covers the behaviour.

We already open-sourced the suite-side cousin—Semantic Graph for Playwright folders. Semantic Canvas brings the same instinct into the product graph: not just tests vs tests, but cross-type meaning across your QA entities.

Same underlying entities. Same link rules. Two complementary ways to see the graph.

The sidebar is where the work happens​

The canvas orients you. The detail sidebar decides.

Select a node and you get accordion lists ranked by cosine similarity—not by how close the dots look after UMAP squashed the space into 2D (those can disagree; trust the scores for linking).

Tune a cosine cutoff per source→target pair. Raise it when you’re hunting true duplicates; lower it when you’re fishing for coverage gaps.

Then act:

• Link — same model as InfiniTrace (including story↔scenario mappings and scenario annotation injection for scenario↔test)
• Mark as distinct — teach the system “similar ≠ duplicate”
• Create issue for duplicate — turn a lookalike pair into cleanup work
• View details — read-only modal, then jump to the full page when you need to edit

If a card that looks nearby on the map falls below your cutoff, that’s not a bug—that’s the difference between a 2D sketch and embedding space. The sidebar keeps you honest.

Borders that mean something​

Pretty clusters aren’t enough. You need health overlays.

Per type, paint node borders with operational signal:

• Latest execution status on tests / scenarios
• Priority / severity and due status on stories, scenarios, issues
• Has linked test on scenarios
• Has test coverage on TrueCoverage events

Drop Stories + Scenarios + Tests with execution status on the tests, and the map stops being art: requirement neighborhoods with live automation health.

Or load Issues alone with severity borders—and watch lookalike bugs collapse into clusters you can actually triage.

Combine that with folder filters and status chips for sharper triage: failing tests next to the stories they should cover, severity islands of duplicate bugs, uncovered hot events next to automation islands.

From links to neighborhoods​

Dashboards tell you what’s trending.

Atlas tells you where in the product structure something lives.

TrueCoverage tells you what real users do versus what tests cover.

InfiniTrace tells you how the pieces connect via explicit edges.

Semantic Canvas tells you what belongs together by meaning—even when the link was never created—and where quiet duplicates and near-misses still hide.

That’s the difference between knowing coverage is “82%” and seeing the blob of scenarios that are really one idea wearing three costumes.

Go open QA Brain → Semantic Canvas. Load Stories, Scenarios, and Tests with execution status—or Issues with severity—and click something you think is unique. If the sidebar shows a 90% twin you forgot existed… good. That’s the product working.

Full walkthrough: Semantic Canvas docs.
Pair it with: InfiniTrace.

TL;DR: QA revolves around a handful of core entities—stories, scenarios, tests, executions, issues, releases, screens—and they aren’t isolated: stories are verified by scenarios, scenarios by tests, tests produce executions, issues trace back to failures. For most teams, that knowledge lives scattered across disconnected tools, so following the thread means hopping platforms. TestChimp already consolidates those entities and relationships in one place. InfiniTrace (in QA Brain) turns that graph into an infinite board chain: pick a start, fan out to related types, and walk Linked vs Non-Linked entities—including semantic near-misses you can link in place.

The graph was always there​

Think about a single question a QA lead asks fifty times a week:

“What actually covers this story—and what didn’t we link yet?”

Or:

“This production event is hot in TrueCoverage. Which tests emit it?”

Or:

“This ExploreChimp journey found a bug. How does it connect back to plans and automation?”

Those aren’t dashboard questions. They’re traversal questions.

Traditionally, answering them meant hopping across Jira, a test runner, a coverage tool, an issue tracker—each holding a slice of the story, none holding the whole graph. TestChimp already consolidates those entities and their relationships in one platform: entity links, scenario annotations in SmartTests, SmartTest event emissions, exploration → journey structure, release and run context. The knowledge graph was real. What was missing was a way to walk it.

We’ve been cooking something for that.

Introducing InfiniTrace​

InfiniTrace is a horizontal link explorer for your project’s QA entities.

Open QA Brain in the sidebar → InfiniTrace tab.

You get boards—columns—like a kanban that never pretends to be a static board. Each board is an entity type. Board N shows what’s related to the card you selected on board N−1.

1. Choose where to start — Story, Scenario, Test, Issue, Event, Release, …
2. Select a card → it lifts; floating CTAs show related types you can fan out to
3. The next board fills with Linked entities (structural edges)
4. Open Non-Linked for top-N peers—often ranked by semantic proximity
5. Keep walking. Boards grow to the right. Change a mid-chain pick and the obsolete boards fall away

Deep links encode the whole chain in the URL (?locator=), so “look at this path” is a share, not a screenshot.

Docs: InfiniTrace.

Linked vs Non-Linked is the point​

Structural links are great when they exist.

The interesting work often sits in the gap: things that should be linked, or are nearly the same idea in embedding space, but aren’t wired yet.

That’s why every follow-on board has two panes:

From Non-Linked you can Link (when rules allow) or Create issue—without losing the chain you were walking.

Some edges stay derived (for example TEST ↔ EVENT from RUM / SmartTest emissions). InfiniTrace shows them; it doesn’t pretend every edge is a manual click.

Why this matters for agents too​

Humans aren’t the only consumers of this graph.

The same related-entity query model is what we want agents to use via CLI / MCP later: “give me type X related to entity Y, ordered by this strategy, filtered like this, top N.” InfiniTrace is the human-shaped surface of that API.

If you’ve been following our build-in-public arc—policy-traceable workflows, TrueCoverage, plans-as-code—the theme is the same: don’t hide the structure agents need in chat history. Put it in durable, queryable form.

InfiniTrace makes that structure visible and walkable for people first.

What you can actually do with it today​

A few chains that already earn their keep:

• Story → Scenario → Test → Execution — coverage and run health in one pass
• Event → Test — TrueCoverage hotspots meet automation identity
• Release / Test run → scenarios & executions — release triage without ten tabs
• Exploration → Journey → Issues / Tests — exploratory findings with lineage

Filter, search, sort per board. View details in a read-only modal. Jump to the full entity page when you need to edit.

From scavenging to tracing​

Dashboards tell you what’s trending.

Atlas tells you where in the product structure something lives.

TrueCoverage tells you what real users do versus what tests cover.

InfiniTrace tells you how the pieces connect—and where the graph is still incomplete.

That’s the difference between staring at metrics and actually following the thread.

Go open QA Brain → InfiniTrace, pick a story or a hot event, and walk it. If the Non-Linked pane surprises you with a near-miss that should have been linked months ago… good. That’s the product working.

Full walkthrough: InfiniTrace docs.

TL;DR: Loops are winning in prompting because you can trace outcome quality back to the prompt and improve it. We’ve deconstructed agentic SDLC work into <workflow> + <task description> + <behaviour guidance>, catalogued ~12 modular QA workflows (plus composites like run-qa and upkeep), and put the behaviour piece in version-controlled policies in your repo. Every workflow execution records policy file · version · git SHA—so outcome data can feed back into the policy. From ad-hoc gut-feel prompting → traceable, structured, modular workflows.

Loops win when you can close them​

Loops are all the rage in prompting today—and rightly so.

If you can trace the quality of an agent’s work back to the prompt, you get a feedback loop.

Better prompts → better outcomes.

We’ve been cooking something along those lines. But instead of treating every agent invocation as a raw blob of prompt text, we deconstructed agentic interactions a bit first.

Most agent work is three parts​

Look carefully at how agents are actually used in the SDLC. Most interactions are of the form:

<workflow> + <task description> + <behaviour guidance>

What’s interesting is that the first part has a surprisingly small vocabulary.

Author Story. Implement Task. Write Tests. Fix Issue. Fix Test Failure…

For QA, we found these distill into around 12 core workflows—plus a couple of composites for shorthand (run-qa, upkeep). The Workflows catalog is that vocabulary: modular playbooks the skill and the platform share.

From there, outcome quality mostly depends on two things:

Today, both still happen ad-hoc—with gut feel. Someone pastes a long prompt, tweaks a line that “felt” important last time, and hopes the next run is better. There is no durable artifact to improve. There is no evidence trail tying this outcome to that guidance.

We already hardened the task side​

We’ve been attacking the task-description lever for a while:

• DeFOSPAM requirement quality checks — score and fix ambiguous specs before agents spend tokens implementing or automating them
• Rich context for failures — fix-test-execution and related workflows pull execution detail instead of “tests are red, please fix”
• Test Planning as Code — stories and scenarios as markdown in Git, so the task itself is structured and agent-readable

That closes one half of the loop: better inputs into the agent.

The other half—how the agent should approach the work for your team—was still mostly vibes in a system prompt.

Introducing policy-traceable workflows​

Today we’re making that second lever first-class: policy-traceable workflows.

Each catalog workflow is backed by an optional policy—a Markdown file that defines how the agent should approach the task for your project. Policies are not secret sauce buried in a chat window. They live in your repository:

plans/knowledge/policies/*.policy.md

Version-controlled. Reviewable in PRs. Shared team-wide. Same files local /testchimp runs and Automations use.

---
workflow-id: implement
version: 1.2.0
---

Playbooks stay generic (the battle-tested skill steps). Policies hold the project choices: scoping rules, environment strategy, which composite subflows to run or skip, quality bars, exclusions, domain quirks.

Author or refresh them with /testchimp create policy <workflow-id>. Bump version whenever guidance changes—that version is what makes the loop measurable.

Full model: Workflows · Policies.

Trace every execution to policy · version · SHA​

A policy you can’t attribute is just another prompt.

For Plan → approve → Execute runs, the agent mints a stable workflow_execution_id, then reports mutative actions with:

• which policy file was used
• the policy version
• the git SHA it came from

…alongside workflow id, actor, branch, and entity identity. That lands on the Workflows execution timeline in TestChimp—not only in chat history that evaporates when the session ends.

The result is data you can use to improve policies. Data you can feed to an agent alongside the existing policy—to iterate on it.

Instead of tweaking prompts based on gut feel, you iterate on policies using evidence.

Ad-hoc prompting → modular, traceable workflows​

Put it together:

1. Small vocabulary of workflows — implement, create-tests, fix-issue, fix-test-execution, run-explorechimp, … plus composites
2. Context-rich tasks — governed requirements, execution detail, scoped branch diffs
3. Versioned behavioural guidance — *.policy.md in Git
4. Attribution on every run — policy file + version + git SHA on the execution timeline
5. Evidence → next policy version — close the loop

That is the same philosophy as skills as SaaS distribution (the playbook travels with the agent) and boiling the QA lake (agents in a continuous feedback loop)—applied to the behaviour contract itself.

From ad-hoc prompting → to traceable, structured, modular workflows.

Frequently asked questions​

What is a workflow policy?​

A policy is a project-owned Markdown file (plans/knowledge/policies/*.policy.md) that tells the agent how to run a catalog workflow for your team—scoping, env, quality bar, composite subflows, exclusions—without rewriting the skill playbook.

How is this different from a system prompt or CLAUDE.md?​

Those are useful, but usually opaque and hard to A/B. Policies are per-workflow, semver’d, synced to the platform, and recorded on every execution with file name, version, and git SHA—so you can compare runs and improve the guidance with evidence.

Do I need a policy for every workflow?​

Defaults are seeded for common composites (run-qa, upkeep) on /testchimp init. Some workflows (notably connect-to-test-env) need an explicit policy before dependent automation stays healthy. Atomic workflows can fall back to broader instructions—but named policies are what make optimization and auditability real.

How do I try an alternate policy without changing the default?​

Keep implement.policy.md as the team default and pass a variant:

/testchimp implement US-181 --policy implement-strict.policy.md

Same playbook, different behavioural contract. Compare outcomes on the execution timeline.

Where do I see policy traceability?​

Executions → Workflow Executions (and the Workflows UI timeline). After mutative actions, reported runs show the policy file and version used for that execution.

Try it​

1. Install the TestChimp skill and map plans/tests in Git
2. Run /testchimp init (or author policies with /testchimp create policy <id>)
3. Open plans/knowledge/policies/*.policy.md, encode one team convention, bump version
4. Run a workflow (/testchimp implement …, /testchimp run QA, …)
5. Check Workflow Executions for policy file · version · SHA—then improve the policy from what you see

Start here:

• Workflows overview
• Create policy
• Automations (same policies, event-driven)
• TestChimp skill

Better policies → better outcomes. Now you can prove which version got you there.

Further reading​

TestChimp

• Workflows
• Create policy
• Run QA
• Upkeep
• Implement Requirement
• Automations

Related posts

• Put QA on Auto-Pilot with Workflow Automations
• Requirement quality governance (DeFOSPAM)
• SKILLs as SaaS distribution
• Boiling the QA lake
• Test planning as code

Concepts

• Agent Skills
• TestChimp skills repository

TL;DR: We've shipped Automations in TestChimp—event-driven runs of the same catalog workflows you already invoke with /testchimp. When an issue is created, a story moves to ready, a CI batch fails, or a release lifecycle status changes, TestChimp can hand the work to a cloud agent—today via ChimpHands (recommended), a labelled GitHub Issue, or a webhook. Aggregation collapses noisy bursts; optional human gates keep high-blast-radius runs under review. Local agents stay for interactive work. Automations cover the complementary case: the platform watches, then starts the agent for you.

The prompt-pasting tax​

If you've been living in the agentic QA loop, you already know the playbooks:

/testchimp fix issue BUG-1042
/testchimp implement US-181
/testchimp fix test execution …
/testchimp run QA

That works. It also creates a new bottleneck: someone has to notice the event and paste the prompt.

Agents compressed execution. They did not invent routing. Automations are that routing layer—on the same workflows, policies, and workflow-execution-id traceability you already use.

Full product docs: Automations.

What is a TestChimp automation?​

An automation is three parts:

Trigger + Action + Config

Policies stay *.policy.md in Git. Automations optionally attach them—the same files local /testchimp runs use. Workflows still define how the agent works. Automations define when that work starts and how the cloud agent is reached.

Project event  →  matching automation  →  aggregation window
                                              ↓
                                    workflow execution queued
                                              ↓
                         (optional) human approve invoke / plan
                                              ↓
                    ChimpHands on CI  or  labelled GitHub Issue  or  webhook
                                              ↓
                         /testchimp <workflow> … --workflow-execution-id …

Ways to reach a cloud agent​

Configure once under Project Settings → Automations (and ChimpHands setup for the native path), then pick a strategy per automation.

ChimpHands (recommended)​

TestChimp dispatches your repo's chimphands.yml GitHub Actions workflow with the rendered /testchimp … prompt. The agent runs on your CI with full checkout context, streams progress to the ChimpHands UI, and opens a PR when done.

One-click setup: GitHub App + workflow file + TESTCHIMP_API_KEY secret. See ChimpHands and Meet ChimpHands.

Via GitHub Issue (bring-your-own agent)​

TestChimp opens an issue on your mapped repo. Body = the full /testchimp … prompt (plus the workflow execution id). Labels always include testchimp, plus any extras you configure.

Point Copilot coding agent, Codex, Cursor Cloud, or your own runner at those labels. The agent checks out the repo, runs the prompt with the TestChimp skill + MCP, and reports back—so Executions → Workflow Executions stays current.

Deep dive: Cloud agents for automations.

Aggregation and human gates (because autonomy without brakes is chaos)​

Matching events should not always mean "spawn an agent immediately."

Default rate limit: 5 cloud agent invocations per hour per project (shared). Over-cap runs stay Queued—not dropped.

Two independent human gates (off by default):

1. Approve before invoke — review the rendered task before the cloud agent starts
2. Approve before plan execute — agent plans first; you approve before execute

Pending work surfaces in Executions → Workflow Executions. You can also copy the prompt and run it locally—the workflow-execution-id keeps the timeline accurate either way.

Mechanics: How automations work.

Recipes teams actually want​

Task templates use {{issue.id}}, {{story.title}}, {{batch.failed_count}}, {{project.repo_url}}, and friends—with autocomplete in the wizard.

More starters: Typical automation setups.

Why this matters in the agentic era​

We already argued that skills are SaaS distribution—the playbook travels with the agent. We argued that requirements must be governed before agents spend tokens. We argued that release governance is the ship decision, not a spreadsheet.

Automations close another gap: intent without attendance.

Without them, QA-on-autopilot still needs a human dispatcher. With them, the same /testchimp catalog becomes a control plane: events in → policy-backed workflow executions out → auditable status in the product. Local Claude / Cursor for interactive loops. ChimpHands and other cloud agents for the overnight and the "while you were in standup" cases.

That is how you boil the lake without standing next to the kettle (boiling the QA lake).

Frequently asked questions​

What are TestChimp Automations?​

Automations are event-driven rules that run TestChimp catalog workflows via a cloud agent when something changes in your project—issues, stories, scenarios, test executions, CI batches, or releases—optionally aggregated and gated by human approval.

How do Automations differ from Workflows?​

Workflows define how an agent should work (skill playbook + optional policy). Automations define when that work starts and which cloud invocation path to use. Both share the same execution timeline and workflow-execution-id reporting.

ChimpHands vs GitHub Issue—which should I pick?​

Use ChimpHands for the default path—native sessions on your GitHub Actions runner, platform chat, and QA-metered credits. Use GitHub Issue when you must keep an external label-watching agent (Copilot, Codex, custom bots) and already have the TestChimp GitHub App.

Will Automations spam my agents on every CI flake?​

Not if you configure them well. Prefer Buffered aggregation for batches, exclude ExploreChimp reporters on auto-fix rules, use conditions (status = BATCH_INVOCATION_FAILED, severity = HIGH), and keep the project hourly invocation cap. High-risk workflows should start with human gates on.

Can I still run workflows locally?​

Yes. Automations don't replace /testchimp in your IDE. From a pending execution you can also copy the prompt—including the workflow execution id—and run it with a local agent so status still updates in TestChimp.

Where do I create and monitor Automations?​

Create under Workflows → Automations (or contextual Automations CTAs on Issues, Plans, batches, releases, and workflow pages). Configure strategies and rate limits in Project Settings → Automations. Monitor and approve under Executions → Workflow Executions.

Try it​

1. Connect GitHub and set up ChimpHands (or configure GitHub Issue / webhook under Project Settings → Automations)
2. Open Workflows → Automations → Create Automation
3. Pick a trigger (start from a popular condition template)
4. Choose a workflow, task template, policy, and invocation strategy
5. Leave Buffered aggregation on; add invoke or plan-execute approval for risky jobs
6. Trigger a matching event—then watch Workflow Executions

Start here:

• Automations overview
• ChimpHands
• Create an automation
• Cloud agents
• Typical setups
• Workflows
• TestChimp skill

When something changes, the agent should already be working—not waiting for you to paste a prompt.

Further reading​

TestChimp

• Automations
• ChimpHands
• How automations work
• Cloud agents
• GitHub integration
• Fix issue
• Implement Requirement

Related posts

• Meet ChimpHands
• SKILLs as SaaS distribution
• Requirement quality governance
• Release governance
• Boiling the QA lake
• Test planning as code

Concepts

• GitHub Copilot coding agent
• Agent Skills