Getting started

Choose the version

These instructions describe the 0.2.0-beta.1 candidate. On September 27, 2026 (UTC), the public npm beta tag still resolved to 0.1.0-beta.3. Do not assume that tag installs the version shown in the demo. Check npm view @asserthive/webtest-ai dist-tags and npx webtest-ai --version.

Before publication, request the candidate .tgz from the maintainer and use the isolated beta trial. The private source repository is not required. After publication, install the explicit version shown below.

Install after publication

mkdir webtest-trial
cd webtest-trial
npm init -y
npm install --save-dev @asserthive/webtest-ai@0.2.0-beta.1
npx playwright install chromium

After installation, npx webtest-ai resolves the local CLI. Pin the package version so the example and runtime stay aligned.

Try the demo

npx webtest-ai demo

This runs twelve Markdown goals against the bundled Everyday Goods shop in isolated browser contexts. The default mode uses deterministic navigator and selector fixtures. It verifies the runtime and reports; it does not measure live model capability.

For an external agent such as a Codex session:

npx webtest-ai demo session headed

The command prints a control-file path. The external agent reads each fresh request and supplies an action through the packaged demo-site/sessionControl.js helper. See the session workflow for the complete instructions. There is no automatic navigator fallback.

For Ollama or a hosted model, see live model testing.

Write a goal

Create specs/homepage.md:

---
suite: homepage
baseUrl: https://example.com
tags: [smoke]
---

# Homepage loads

## Goal
Open the homepage and check its heading.

## Success
- Text: Example Domain

Goal tests require a model config with agent.profile. Copy one of the packaged examples/config/goal-agent*.json templates into your project and configure the endpoint and model. Store keys in environment variables named by apiKeyEnv.

npx webtest-ai run --suite specs/homepage.md --config webtest-ai.config.json

For an exact test without a model, replace the Goal and Success sections with:

## Steps
1. Open "/"

## Expected
1. Assert text "Example Domain"

Both forms use the bounded engine. Explicit browser assertions decide the result.

Review results

Reports are written under artifacts/reports/; per-test screenshots and observations live under artifacts/<runId>/<testId>/.

• HTML: verdicts, action screenshots, model identities, usage, cost status, accessibility findings, and lab LCP/CLS.
• Agent JSON: structured results and quality statuses for agent workflows and CI.
• Calls NDJSON: one record per tool or model call.

See Markdown goals, demo modes, and the migration guide.

See limits, multi-page journeys, and login reuse before configuring longer authenticated flows.