Published Updated
What is TestChimp?
In brief: TestChimp gives agent-driven teams release confidence. It measures your QA posture and runs a feedback loop with your coding agents, so QA keeps course-correcting as the product changes. Humans set the product intent and oversee. Agents do the work.
More tests. Yet less confidence.
Agents now write the code and the tests. Suites grow fast, but the number of tests is often the only assurance a team has left. Nobody can say:
- which parts of the product are tested,
- where the gaps are,
- whether a person has checked that the tests verify anything.
TestChimp makes those answers measurable. Your coding agents read the measurements and close the gaps, and people see the evidence before they ship. What you measure, agents can now improve, continuously.
It supports web (Playwright) and native mobile (iOS and Android through Mobilewright). See Mobile testing for platform specifics.
How it works
- Capture intent. User stories and test scenarios live as Markdown specs in a
plans/folder in your repo, synced both ways with TestChimp. MeetBots turn decisions from calls into spec updates, and AgentWatch picks up requirement changes from coding-agent conversations. You review and approve them. - Write tests. When a feature branch is done, run
/testchimp run QA. Your agent writes or repairs E2E tests against the specs, each one traced to its scenario. Tests stay plain Playwright, instrumented by the TestChimp fixture so they can be measured. See Run QA. - Measure posture. TestChimp measures what the suite really covers: requirement coverage, API schema coverage, TrueCoverage (real user flows from production) and UX checks by ExploreChimp, all reported in Atlas.
- Feed gaps back to agents. On your release cadence, run
/testchimp upkeep. It finds requirement and TrueCoverage gaps, hands them to your agent to close, and cleans redundant tests out of the suite. See Upkeep. - Ship with evidence. Run
/testchimp create release. Release health combines functional and non-functional checks, and shows which tests a person has verified and which went stale since. People review, approve and govern. See Create Release and Release readiness.
What gets measured, and how
Tests stay plain Playwright in your repo and run anywhere Playwright runs. Specs import test from a fixtures file that wraps Playwright with the TestChimp fixture, and the @testchimp/playwright reporter sends results back. That is what makes posture measurable:
| Signal | How it is captured |
|---|---|
| Requirement coverage | Playwright annotations link each test to its scenario. |
| API schema coverage | The fixture captures API payloads during the run (real and mocked responses told apart) and maps them to your OpenAPI specs. |
| TrueCoverage | The TestChimp RUM SDK captures user events in production and in test runs (web, iOS, Android), tagged per test by the fixture. |
| Screen states | markScreenState, the only TestChimp line in a test body, feeds ExploreChimp and Atlas. |
Details: Playwright runtime plugin and Instrumenting your app.
Where it runs
The TestChimp web app is the control plane: plans, coverage and gaps, bugs found by ExploreChimp, TrueCoverage drill-downs, workflow runs and release health. TestChimp Studio is the same control plane as a desktop app, with a local agent, manual testing capture and AgentWatch. Download: TestChimp Studio.
Quick start
- Connect your Git repo in TestChimp and map two folders: one for tests and one for plans.
- Install the TestChimp skill on your agent host (Claude Code, Cursor, Codex and others) and register the MCP server.
- Run
/testchimp project-initonce per project, and/testchimp initonce per teammate. - Run
/testchimp run QAwhen a feature branch is done,/testchimp upkeepon your release cadence, and/testchimp create releasewhen you ship.
Step by step: Quick Start. Team and project admin: Onboarding to TestChimp. Composites, atomic workflows and policies: Workflows.
Why not "just use Claude"?
Claude writes tests. TestChimp tells it which ones matter.
You should let Claude (or Cursor, or Codex) write your tests. It is fast and it is good at it. But writing tests is not the same as knowing you are covered. Once an agent is producing tests at speed, a different set of problems shows up, and none of them are solved by writing more tests.
TestChimp is the QA layer above test writing. It measures your QA posture in detail, so your coding agents know exactly where the gaps are and can close them. What you can measure, you can improve.
The problems teams hit after the agent writes the tests
| What happens | Why the agent can't fix it alone | What TestChimp adds |
|---|---|---|
| "Are our requirements actually tested?" Nobody can answer without reading every spec file. | The agent only knows what is in its prompt. Decisions made in meetings and earlier chats never reach it. | Requirement coverage. Stories and scenarios live as markdown in your repo, tests link to them with a #TS- annotation, and coverage rolls up per folder, release and environment. MeetBots and AgentWatch keep the plans current from meetings and coding-agent sessions, so tests are aimed at what the product should do now. |
| "How much of our API surface is tested?" The suite is green, but whole endpoints and response codes were never exercised. | The agent sees the code it touched, not the whole contract. | API schema coverage. Per-operation, field and response-code coverage from your OpenAPI specs and captured test traffic, with gaps you can hand straight to an agent. |
| "Are the flows our users actually use tested?" Tests cluster around whatever was built last. | The agent has no view of production behaviour. | TrueCoverage. The same user events are captured in production and in test runs (web, iOS and Android), so you see which real user pathways have no tests, ranked by demand and drop-off. |
| "What should we test next?" | Without signals, every gap looks equally important. | The measures above feed /testchimp run QA, /testchimp upkeep and /testchimp create release, so agents work through the highest-value gaps first instead of adding more tests in already-covered areas. |
| Tests nobody reviewed. The agent wrote the code and the test, and grades its own work. | An agent can't independently confirm its own reading of the requirement. | Verified Tests. A person on the team inspects the run and marks the test verified. When an agent later edits that test, it is marked stale and the original verifier is asked to re-verify. How-to: How do I get humans to review tests my coding agent wrote?. |
| The suite gets big and slow. Every PR runs everything, or someone hand-picks a subset. | The agent has no record of which tests relate to which change. | Smart-Smoke. Impact analysis picks the related tests and an optimal subset of a large suite for each PR. |
| Suites drift and go flaky. Tests were right at the first green run and decay after that. | Upkeep is nobody's job until something breaks. | Upkeep closes coverage gaps and prunes redundancy on your release cadence. Optional natural-language steps (ai.act, ai.verify, ai.extract, web today) resolve intent at run time, so small UI changes don't break scripts. |
| "Are we ready to ship?" The answer comes from a release meeting or a slide. | A green CI run doesn't say what is uncovered or unverified. | /testchimp create release runs release QA per policy and summarises release health. Release readiness shows test runs, requirement coverage and verification status in one picture. |
It works where your agent already works
- Your files, your repo. Tests are standard Playwright (web) or Mobilewright (native iOS and Android) specs. Plans are markdown. Run evidence such as screenshots, traces and video follows your Playwright config. No lock-in.
- Your agent does the work. Install the TestChimp skill in Claude Code, Cursor or another host, then run
/testchimp run QAafter each feature. TestChimp supplies the signals and the workflows, and your agent writes and fixes the tests. - You steer. Agents propose plan and test changes, and the team reviews them as normal pull requests. People verify tests and approve releases.
In short: Claude writes the tests. TestChimp measures what they cover, against your requirements, your API contract and your real users, and points Claude at the gaps that matter. Then it tells you whether the release is ready.
Deep dive
- Workflows: the commands, composites, atomic workflows and policies.
- Requirement Planning: Markdown plans, quality governance, traceability and Git sync.
- SmartTests: Playwright tests on web, with optional natural-language steps
ai.act/ai.verify/ai.extract(web today). - TrueCoverage: coverage of real user behaviour from events in production and tests (web, iOS, Android).
- Release Management: release health, gating and approvals.
- Performance Testing: k6 load and volume journeys with comparable runs.
- Mobile testing: native iOS and Android with Mobilewright.
Compare, alternatives, and guides
- Vendor comparisons: TestChimp vs Selenium, Cypress, Playwright and more.
- Alternatives hub: switching from legacy QA tools.
- Testing guides: QA workflows by industry and by builder (Cursor, Lovable, Claude Code).
- QA in the Age of AI: how AI changes test creation and maintenance.
Get started
- Quick Start: get running in a few steps.
- Onboarding to TestChimp: team and project setup.
- TestChimp Studio: the desktop app. Download for macOS, Windows or Linux, or run
npx @testchimp/studio.
FAQ
What does TestChimp do that Claude alone cannot?
TestChimp tells agents which tests to write and why—using requirement traceability, TrueCoverage gaps, and orchestrated /testchimp init, test, create release, upkeep, and explore commands—not just script generation.
Our team doesn't have QA — can we use TestChimp?
Yes—that is a core use case. TestChimp is a dev-friendly QA platform for lean teams: install the TestChimp skill, run `/testchimp init`, and use `/testchimp test` on every PR so agents write and repair Playwright SmartTests against markdown scenarios. Requirement coverage rolls up from scenario annotations, and `/testchimp create release` drives release QA posture when you ship—so you ship reliable software without building a large QA org first.
What platforms does TestChimp support?
Web via Playwright SmartTests and native iOS/Android via Mobilewright, with TrueCoverage on web, iOS, and Android when instrumented.
How do I get started quickly?
Connect Git, map plans/ and tests/ folders, install the TestChimp skill, then run /testchimp init once, /testchimp test on each PR, and /testchimp create release when you ship.