Suites overview
A suite runs several chains together and gives you a single answer: did it pass?
A chain is one flow. A suite is the layer above it — an ordered set of chains, the setup that prepares the state they need, the teardown that cleans up afterwards, and one aggregated verdict over the lot. It is what turns a collection of workflows into something you can point at a system and trust the result of.
Suites are available on the Engineering tier and up. Designing and saving a suite works on any tier; running one requires the entitlement.
Creating a suite
Section titled “Creating a suite”Suites live in a bundle, like chains and scripts, and appear as leaves in the Project Explorer tree. Use the ＋ menu in the Explorer header (Add Suite) to create one; it lands in the bundle you have selected, or in the active project’s Default bundle when nothing is selected, and opens as an editor tab.
The three phases
Section titled “The three phases”A suite has three ordered lists of chains. Each one is picked from the chains visible in the active project, and each is a reference: the suite links its chains, it never copies them. Edit a chain and every suite that uses it runs the new version — the same live-versus-frozen rule as a linked chain step.
| Phase | When it runs | What it is for |
|---|---|---|
| Setup | First, in order | Build the state the tests need: create the account, seed the record, obtain the token |
| Tests | After setup | The actual checks |
| Teardown | Always, last | Clean up what setup created |
What the phases guarantee
Section titled “What the phases guarantee”These rules are the reason a suite is more than “run these chains in a row”:
- Setup output feeds the tests. Setup chains run in sequence, each seeing the previous one’s variables, and the resulting context is what every test chain starts from. That is what makes a fixture useful: setup creates the record, the test asserts on its id.
- Test chains are isolated from each other. Each starts from a copy of the post-setup context, never from the previous test’s output. No test can quietly depend on another, and reordering them changes nothing.
- Teardown always runs, even after a failure, and a failing teardown does not turn a passing suite red. Cleanup noise must not mask a green test run.
- A failing setup skips the tests. Testing against a broken fixture produces confidently wrong results, which is worse than no result at all. Skipped chains are reported as skipped, never as passed.
Stop at the first failure
Section titled “Stop at the first failure”Off by default: every test chain runs and the suite reports an aggregated verdict, which is what you want from a test suite. Turn it on when a later test is pointless once an earlier one has failed.
Running a suite
Section titled “Running a suite”Run (Ctrl+Enter) executes the suite in the background and streams progress back live: each chain is marked running, passed or failed as it happens, with the step currently executing shown alongside. You are not waiting on a spinner with nothing behind it.
When the run finishes, the report shows every chain with its phase, result and detail, the variables setup produced, and the aggregate verdict — chains passed, assertions passed.
Evidence and reports
Section titled “Evidence and reports”- JSON evidence — the full result of the run, exported client-side from what you already have on screen. Nothing re-runs, so the evidence is the run you actually saw.
- JUnit XML — the suite as a CI runner expects it: one testsuite per chain, its steps as testcases, skipped chains as skipped testcases. “Not run” and “passed” never look the same.
- Run history — the last 20 runs of each suite are kept. Click one to bring its report back.
Test health on the dashboard
Section titled “Test health on the dashboard”The dashboard opens with the health of your tests rather than a count of what you own: the pass rate over recent runs, the latest chain and suite runs, and what regressed.
A regression is deliberately not “the last run failed”. Something that has only ever failed is unfinished work; listing it would bury the runs that actually changed behaviour. Only targets that passed before and fail now are reported, with how long they have been failing and when they last passed.