webtest-ai run

Runs one Markdown suite file or a directory of suite files through the WebTest AI parser, interpreter, execution driver, and reporters.

webtest-ai run --suite specs/homepage.md
webtest-ai run --suite specs --tags smoke
webtest-ai run --suite specs/checkout.md --target brand-b-fr
webtest-ai run --suite specs --exclude-tags serial --workers 4

The candidate defaults to bounded execution and native CDP Chromium. Goal specs need agent.profile; exact-step specs can run without navigator inference. A custom target must exist in config.targets. Run from your own project, not inside the installed package.

Trace and journey flags are retained for legacy execution; they do not enable those features in the bounded engine. Non-Chromium browser aliases require a compatible driver and are not covered by the bounded Chromium validation.

Flags

• --suite <path>: Markdown suite file or directory. Defaults to specs/.
• --config <path>: JSON config path. Defaults to webtest-ai.config.json.
• --browser <name>: browser override. Supported aliases are chromium, chrome, google-chrome, edge, msedge, firefox, webkit, and safari.
• --target <a,b> / --targets <a,b>: runs selected configured targets from config.targets.
• --tags <a,b>: runs tests containing all listed tags.
• --exclude-tags <a,b>: skips tests matching any listed tag.
• --workers <n>: runs parallel-safe tests concurrently. Tests tagged serial remain serial.
• --headed: opens a visible browser.
• --debug: requests headed debugging. Playwright Inspector requires the Playwright driver and legacy execution; bounded execution does not pause in Inspector.
• --refresh-auth: ignores reusable auth state and creates a fresh session when the suite uses auth.
• --auth-profile <name>: overrides the auth profile used for reusable sessions.
• --profile fast|ci|debug: applies a runtime profile overlay before other run flags.
• --trace <mode>: overrides trace capture, for example off, on, always, or retain-on-failure.
• --screenshot <mode>: off, always, only-on-failure, or every-step (goal execution).
• --quality off: disables a11y and vitals checks for this run.
• --journey <mode>: captures journey snapshots with navigation or step; use off to disable.

Output

The command prints suite counts, status counts, run ids, and report paths. JSON and HTML reports are written under artifacts/reports/; per-test artifacts are written under artifacts/<runId>/<testId>/.

The process exits nonzero for failed, inconclusive, or blocked results, and for command/configuration errors. Inspect skipped counts separately; a successful process exit is not proof that every selected test executed.