Suites and Play All | ReplayQA Docs

A suite is a named set of approved tests, and whole folders of tests, that you run together. Most teams start with a quick Smoke suite and add one per area, such as Checkout, Account and Admin.

Create a suite

Open Suites (Play All) in the sidebar and press New suite.
Give it a name and, optionally, a description, for example "Run before every release".
On the suite page, open Add tests. Add whole folders from the list at the top, or search tests by name or tag, tick the ones you want (or Add all matching), and press Add selected. Only approved tests are listed unless you turn on Include drafts; drafts in a suite are marked Skipped when played until they're approved.
Press Save changes in the bar at the bottom of the page.

Folders in a suite

Adding a folder adds every test in it and in its subfolders, as they are each time the suite is played. File a new test into the folder and the suite runs it next time, with no change to the suite. Archived tests in the folder are left out.

The suite runs its list from top to bottom. A folder's tests run in its place: the folder's own tests in the folder's order (A–Z unless you set one), then each subfolder's.
A test that's in the list twice (on its own and through a folder) runs once, at its first place.
The Tests count and ready to run include the tests the folders add, without counting a test twice.
If you delete a folder that a suite includes, the suite gets that folder's tests as individual tests in its place, so it keeps running the same tests.

To file tests into folders, see Organize tests in folders.

Managing a big suite

The suite page has four sections: Tests, Add tests, Run history and Settings.

Filter the Tests list by name or tag, or show only tests that are ready to run or will be skipped.
Reorder by dragging a test's handle, using the arrow buttons, or focusing the handle and pressing the up and down arrow keys.
Select several tests or folders to move them to the top or bottom, or remove them together.

Changes to the test list are kept on the page until you press Save changes (or Ctrl+S). Discard puts the list back as it was. Play All runs the saved suite, so save first. A suite runs up to 1,000 tests, counting the tests its folders add.

Suite settings

Setting	What it does
Default site	The site the suite runs on unless you choose another when you press Play All.
Tests at the same time	How many tests run at the same time (up to your plan's limit). Higher is faster; lower is gentler on a small staging server.
Run in this order	Runs tests one after another in the listed order, for flows that build on each other.
Stop on first failure	Skips the remaining tests as soon as one fails, which is useful for a quick smoke check.

Open Settings, make your changes and press Save settings. The button is available once something has changed, and Undo changes puts the form back.

Prefer independent tests that can run in any order. They're faster in parallel, and one failure doesn't cause a cascade. Use Run in this order only when a flow really depends on a previous one.

Run it

Press Play All. A progress page shows each test's status as it runs, with totals for passed, failed, running and skipped. You can leave the page and come back; the batch keeps running.

Retry N failed re-runs only the tests that failed, which helps separate real failures from one-off glitches.
Cancel all stops every queued and running test in the batch.

Recent Play All runs are listed on the suite page, so you can see whether a suite was green before a release.

Running everything

You don't need a suite to run many tests: on Tests, tick the box at the top of the list to select every approved test, then press Run selected.

Tests over your plan's limit

If your workspace has more saved tests than its plan allows (for example after a downgrade), Play All runs the tests within the limit and marks the rest Skipped with the reason. See Plans, usage and limits.