Skip to main content

Published

Spec-Driven Testing: How to Test Against Your Spec (2026)

Short answer

Spec-driven testing means every test traces back to a statement in a written spec, and every spec statement has at least one test. It is the testing half of spec-driven development, where a coding agent works from a structured spec instead of a one-line prompt. The hard part is not writing the tests. It is seeing which spec items have none.

Last updated: 2026-11-11

What is spec-driven development?​

Spec-driven development is an AI-assisted workflow that starts with a structured functional specification and breaks it into smaller pieces, designs and tasks. Thoughtworks put it on its Technology Radar in the Assess ring in November 2025 and noted that the definition is still evolving.

Two tools show the common shape. Amazon's Kiro generates three files per spec: requirements.md (user stories with acceptance criteria), design.md and tasks.md. GitHub's Spec Kit runs a sequence of commands, /speckit-specify, /speckit-plan, /speckit-tasks and /speckit-implement, with an optional /speckit-analyze step that reports conflicts and gaps across the spec, plan and tasks. Spec Kit also has a "constitution" of principles the agent must follow.

Both put the spec first and the code second, so the spec is the natural source for what to test.

What is spec-driven testing?​

It is the practice of deriving tests from the spec and keeping a live link between the two. Kiro writes acceptance criteria in "GIVEN... WHEN... THEN..." form, which maps directly onto a test: set up state, perform an action, assert the result. The Cucumber documentation makes the same point about Gherkin: an example is also a test, so your examples form an executable specification.

In practice, spec-driven testing has three parts:

  1. Each spec item gets an identifier. An acceptance criterion or scenario that you cannot point to cannot be linked to a test.
  2. Each test names the item it checks. The link lives next to the test, in code, so it survives refactors.
  3. Someone can list the spec items with no test. This report is what turns a spec from documentation into a coverage target.

How do I turn a spec into tests?​

Start from the acceptance criteria. Here is one criterion in Given/When/Then form:

Scenario: Expired reset link
Given a user with a password reset link older than 30 minutes
When they open the link
Then they see a message that the link has expired
And a button to request a new link

The matching Playwright test follows the same three beats:

tests/password-reset.spec.ts
import { test, expect } from '@playwright/test';

test('expired reset link shows request-new-link button', async ({ page }) => {
// Given: a reset link older than 30 minutes (seeded by a test-only route)
const link = await seedExpiredResetLink();

// When: the user opens it
await page.goto(link);

// Then: the expiry message and a new-link button appear
await expect(page.getByText('This link has expired')).toBeVisible();
await expect(page.getByRole('button', { name: 'Request a new link' })).toBeVisible();
});

Ask your coding agent to write this from the spec, then review it. Two habits help. Have the agent quote the criterion in a comment so the reviewer can compare them. And write the negative cases from the spec too: the spec is where "what should happen when it fails" was decided.

Where do spec-driven workflows leave gaps?​

The agent that implements a task also tends to write the tests, and it reads the same spec. That causes four problems in practice.

  • Silent omissions. The agent covers the criteria it noticed. Nothing reports the ones it skipped.
  • Mirrored assertions. A test that asserts whatever the code returns agrees with the code by construction.
  • Spec drift. Kiro's own guidance describes updating requirements and then syncing tasks, which is a manual loop. When the spec changes and the tests do not, the link goes stale.
  • No view across specs. Each spec is one feature. Nobody sees coverage across all of them.

The Thoughtworks Radar entry and Birgitta Böckeler's write-up on Martin Fowler's site raise a related concern: some tools generate long spec files that are hard to review. Long specs mean more criteria, which means more places for a missing test to hide.

How do I check which spec items have no test?​

You can do a rough version with no tooling. Keep scenarios in Markdown in the repo with stable IDs, annotate each Playwright test with the ID it checks, and write a script that lists IDs with no matching annotation.

TestChimp does this as a product feature. Scenarios live as Markdown files in your repo. A Playwright annotation, { type: 'scenario', description: '#TS-<n>' }, links a test to a scenario, and coverage views show which scenarios have tests, which tests are unlinked, and which tests a coding agent has edited without review. See how to trust AI-written tests for the verification side, and Playwright MCP explained for how agents drive the browser while writing them.

Spec Kit's /speckit-analyze checks consistency between the spec, plan and tasks before implementation. Coverage reporting runs afterward and asks whether the tests that exist cover what the spec says.

Frequently asked questions

What is spec-driven testing?

It is testing where every test traces to a statement in a written spec and every spec statement has a test. It is the testing counterpart of spec-driven development, where coding agents work from structured specs.

How is spec-driven development different from TDD?

TDD starts from a failing test written by the developer. Spec-driven development starts from a structured specification that an AI agent turns into a plan, tasks and code. The two combine well: acceptance criteria in the spec become the tests.

What tools support spec-driven development?

Amazon Kiro generates requirements, design and task files per spec. GitHub Spec Kit provides a command sequence with a constitution of project principles. Tessl Framework, per Thoughtworks, goes further and makes the spec the maintained artifact instead of the code. Thoughtworks put spec-driven development itself in the Assess ring of its Radar.

Can I use Given/When/Then acceptance criteria as tests?

Yes. Each Given/When/Then scenario maps to setup, action and assertion in a test. With Cucumber the scenario itself is executable; with Playwright you write the test and keep the scenario text in a comment or linked file.

How do I know which spec items are untested?

Give each spec item an ID, annotate each test with the ID it checks, and report IDs with no test. TestChimp automates this with scenario annotations on Playwright tests and coverage views.

Should the agent that writes the code also write the tests?

It can, but its tests need independent review because the agent reads the same spec and may mirror the code. Record who verified each test and report spec items with no test.

See which specs have no test

Link Playwright tests to scenarios in your repo and see the gaps your coding agent left. 14-day free trial, no credit card.

Start a 14-day free trial · Book a demo