Published
Smoke Test in Software Testing: What It Is, Examples and How to Run It in CI
Short answer
A smoke test is a small, fast set of checks on the most important functions of a build, run to decide whether the build is stable enough for further testing. If the app does not start, nobody can log in or the main page errors, the build is rejected and nobody spends time on deeper tests. Smoke tests are usually automated and run on every build or deploy.
Last updated: 2026-11-16
What is smoke testing?
Smoke testing is preliminary testing that reveals simple failures severe enough to reject a release. It uses a subset of test cases that cover the most important functionality and checks whether the main functions appear to work. Other names for it include build verification test (BVT), build acceptance test and confidence testing.
The term comes from electronic hardware. Kaner, Bach and Pettichord, quoted on Wikipedia, describe powering on a new board: if smoke appears, you switch it off, because there is no point testing further. Software smoke testing keeps the same logic. A broken build is cheap to reject at the door and expensive to discover halfway through a regression run.
A smoke test is wide and shallow. It touches many critical areas but goes deep into none. It answers one question: is this build worth testing?
What does a smoke test check?
Pick flows where a failure would make every other test meaningless or every user unhappy. For a typical web app that means:
- The app starts and the home page loads without a server error.
- A user can sign in, using a seeded test account.
- The main page after login renders its core data.
- The single most valuable action works once: create the record, place the order, send the message.
- A health or version endpoint returns the build you meant to deploy.
- A key third-party dependency, such as the payment sandbox or the database, is reachable.
Keep each check to the happy path. Edge cases, validation errors and permission matrices belong to regression tests. If a smoke suite grows past a few minutes, it has started doing regression's job and people will stop waiting for it.
Smoke testing vs sanity testing vs regression testing
The three terms get mixed up, and sources do not fully agree. Wikipedia treats "sanity testing" as another name for smoke testing. TechTarget draws a line: smoke tests verify the stability of the build across the whole application at a high level, while sanity tests concentrate on specific features and defects, often after a change, and are frequently run by hand.
Regression testing is the broad one. It re-runs a large set of existing tests to confirm that old behavior still works after a change. Smoke testing comes first and gates it.
| Smoke | Sanity | Regression | |
|---|---|---|---|
| Question | Is the build testable? | Does this change work? | Did anything else break? |
| Scope | Whole app, critical paths | One feature or fix | Broad, existing behavior |
| Depth | Shallow | Narrow | Deep |
| Typical form | Automated | Often manual | Automated |
| When | Every build or deploy | After a change | Before release, nightly or per PR |
Teams that use "sanity" and "smoke" interchangeably are not wrong; just write down which meaning yours uses.
What does a smoke test look like in Playwright?
Playwright lets you tag tests. A tag must start with @, and you can set it in the test details or in the title:
import { test, expect } from '@playwright/test';
test('home page loads', { tag: '@smoke' }, async ({ page }) => {
await page.goto('/');
await expect(page.getByRole('heading', { level: 1 })).toBeVisible();
});
test('user can sign in', { tag: '@smoke' }, async ({ page }) => {
await page.goto('/login');
await page.getByLabel('Email').fill(process.env.SMOKE_USER!);
await page.getByLabel('Password').fill(process.env.SMOKE_PASSWORD!);
await page.getByRole('button', { name: 'Sign in' }).click();
await expect(page).toHaveURL(/dashboard/);
});
Then select the tagged tests from the command line:
npx playwright test --grep @smoke
The inverse, --grep-invert @smoke, runs everything else. The Playwright docs also describe a grep option in the config file and in projects, which lets you define a dedicated smoke project that CI can call by name.
Assert something visible and specific. A page that returns 200 with an empty body passes a status check and fails a user.
How do you run smoke tests in CI?
Run them at two points.
On every pull request or merge: Run the tagged subset against a preview or ephemeral environment. It should finish in a few minutes. If it fails, the full suite does not start. See GitHub pull request checks for making the check required before merge.
After every deploy: Run the same tests against the freshly deployed environment, staging or production. A post-deploy smoke test catches problems that only appear in the deployed setup: a missing environment variable, a failed migration, a misconfigured domain. Use a dedicated test account and avoid destructive actions in production, such as real charges.
A few habits keep the suite useful:
- Make failures stop the pipeline. A smoke test that only reports is just a dashboard.
- Keep it deterministic. Flaky smoke tests train people to rerun and ignore them. Seed the data they need rather than relying on whatever is in the environment.
- Review the list quarterly. Remove checks nobody would act on and add the flow that hurt you in the last incident.
- Time it. Set a budget, for example under five minutes, and treat going over as a bug.
Who decides which tests are the smoke set?
The list of critical flows tends to be written once and never revisited, while the product changes weekly. When coding agents write much of the test suite, nobody may even know which tests protect the checkout flow and which exercise a settings page.
The useful input is evidence: which flows real users run most, which areas changed in this pull request, and which scenarios are actually covered by a passing test. Choosing smoke tests by those signals beats choosing them by whoever wrote the first test.
How does TestChimp help?
TestChimp links Playwright tests to written scenarios in your repo and shows which scenarios have verified coverage, so the smoke set can be chosen from known critical flows instead of guesswork. Smart Smoke selects a related subset of tests for a pull request within a time budget; the background is in Smoke vs regression testing, and Smart Smoke for CI.
Frequently asked questions
What is a smoke test in software testing?
A smoke test is a short set of checks on the most critical functions of a build, run to confirm the build is stable enough for further testing. If it fails, the build is rejected without running deeper tests.
What is the difference between smoke testing and regression testing?
Smoke testing is wide and shallow and asks whether the build is testable at all. Regression testing is broad and deep and asks whether existing behavior still works after a change. Smoke tests run first and gate the regression run.
Is smoke testing the same as sanity testing?
Sources differ. Some treat the terms as synonyms. Others, such as TechTarget, say smoke tests check overall build stability while sanity tests focus on specific features or fixes, often by hand. Pick one convention for your team and document it.
How many tests should a smoke suite have?
There is no standard number. Aim for the handful of flows that would make the app unusable if they broke, and set a time budget, such as a few minutes. If it grows past that, move detail into regression tests.
Should smoke tests be automated?
Yes, in most teams. They run on every build or deploy, so manual execution does not scale. A manual smoke checklist is still a reasonable starting point before you automate it.
Can I run smoke tests in production?
Yes, after a deploy, if they are read-only or use a dedicated test account and avoid real charges or customer data changes. Keep them few and safe.
Know which tests protect your critical flows
Link tests to scenarios and see what is verified before you pick the smoke set. 14-day free trial, no credit card.