WebTest AI v2 beta: coming soon
This is preview documentation for the upcoming release. Publication is pending live Jev validation. The demo uses a live navigator with mocked Jev element selection.
Changelog
0.2.0-beta.1 (candidate)
- Wait for frame document commitment before enforcing origin policy; cover delayed frame responses and avoid classifying Chrome’s transient empty frame URL as a forbidden origin.
- Add opt-in hierarchical scoped discovery for large pages, while preserving strict discovery defaults and finite selector choices.
- Support explicit goal frame/tab switching, nested and cross-site frames, and navigation across configured origins; stop denied document redirects and preserve legacy popup timing.
- Add native bounded login validation and refresh with serialized profile preparation, environment credentials, atomic private state files, and redacted authentication telemetry.
- Show authentication source, validation status, execution budgets, and failure guidance in HTML and agent reports.
- Validate 720-control discovery, nested popups, frame input/clicks, blocked redirects, and two concurrent authenticated 15-page journeys in real Chromium.
- Correct candidate installation, goal authoring, command boundaries, and exit-status documentation; add opt-in read-only public-site examples.
- Preserve a goal suite’s starting URL path and query instead of always navigating to the origin root.
- Stop on native CDP navigation errors so connectivity failures do not become misleading content assertion failures.
- Expose accessibility findings and lab LCP/CLS measurements in bounded HTML and agent JSON, including explicit disabled, unavailable, warning, and failure states.
- Add explicit bounded visual checkpoints with reviewed baselines, immutable image evidence, and exact-byte mismatch detection. Missing or newly captured baselines cannot silently pass.
- Make bounded execution the default, with Markdown goals, immutable success conditions, native CDP grounding, and independent browser verification.
- Ship twelve Everyday Goods goal scenarios and the AssertHive theme, with exact-step and fixture modes for repeatable runtime validation.
- Add live navigator profiles for Ollama and OpenAI-compatible endpoints, optional Jev selection, and an external agent session bridge with mocked Jev.
- Report model identities by role, token usage when supplied, explicit cost availability, tool calls, screenshots, observations, and agent-readable JSON/NDJSON.
- Honor configured model deadlines, add CPU-friendly single-worker and smoke modes, and meter Ollama readiness separately.
- Restrict spec frontmatter to YAML/JSON data, reject executable JavaScript engines, and update the vulnerable YAML dependency.
- Drain active workers before shared browser teardown after an unexpected worker error. Validate session deadlines and release failed bridge requests.
- Recheck the allowed origin before element selection, including redirects after the navigator observation.
- Keep tested legacy engine and browser compatibility coverage; remove unused legacy specs, retired workshop assets, and obsolete demo commands.
- Exclude generated reports, recordings, caches, and package archives from source and npm releases. Validate the actual tarball in an isolated consumer project.
Older release history is also available in the repository root CHANGELOG.md.
0.1.0-beta.3
V2 intent-driven AI beta.
- Adds the Bounded Goal Planner for intent-mode goal execution.
- Adds semantic
Assert outcomeassertions backed by model-cited page evidence. - Adds bounded pre-plan page evidence for intent planning, including URL, page fingerprint, visible text, accessibility candidates, and DOM action candidates.
- Adds configured target matrices for whitelabel and translated app variants.
- Validates canonical
Click actionintents against branded demo labels such asLaunch preview/Hide previewacross CDP and Playwright, with state assertions proving the chosen controls executed. - Adds a public intent storefront demo with Northwind, Maison Bleu, and Mercado Claro variants so one goal-based spec can be tested across different brands, themes, locales, and labels.
- Adds a 12-case functional commerce showcase covering account signup, login, collection browse, product search, cart, discount, saved item, profile, newsletter, shipping, and signed-in checkout across all bundled brand/locale targets.
- Adds runnable
demo:intent,demo:discover, anddemo:v2scripts covering commerce, checkout, support, preview-surface, discovery, and UI-inventory proposal demos without requiring a live model server. - Adds
release:check, a publish-workflow demo evidence step, a gated live model matrix, and a local--live-modelintent demo path for beta.3 release hardening. - Adds
Fill intentsteps that only use approved## Dataor environment-backed values. - Tightens field-intent resolution so richer forms with multiple textboxes prefer each field's own label/name/test id before container text.
- Tightens aliased click ranking so localized targets prefer full canonical intents plus configured aliases instead of loose split words that can tie on dense functional pages.
- Adds
--target/--targetsrun selection for configured targets. - Adds
driver.name: "cdp"as a built-in Chrome DevTools Protocol driver selection. - Adds CDP auth-state import/export for reusable cookies and origin storage.
- Adds reviewed intent-plan memory proposals, high-confidence memory reuse before model planning, overwrite reporting, and stale-plan review/pruning commands.
- Adds
Switch page "<alias-title-or-url-part>"for deterministic popup/additional-page selection across CDP and Playwright. - Expands CDP Chromium parity with CDP JSON traces, axe/Web Vitals quality collection, popup target switching, cross-origin frame execution contexts, a text-first accessibility/action surface, wait-style assertions, page switching, and select-label filling.
- Expands discovery so auto exploration can use configured target matrices, reviewed UI inventory context, optional auth-prepared browser-backed page evidence, UI inventory proposal output, and reviewable Markdown spec output.
- Adds CI-friendly intent failure categories and upgrades the gated live model smoke to exercise both
intent.plan_goalandintent.assert_outcome. - Adds a complete
npm run test:v2verification command, fast CDP/Playwright CI smoke coverage, and a bounded publish workflow gate for beta releases.
0.1.0-beta.2
Release workflow hardening beta.
- Bumps the package to the next unpublished beta version.
- Adds a GitHub Actions preflight that fails early when the npm version already exists.
- Switches the beta publish workflow to npm Trusted Publishing through GitHub Actions OIDC.
- Documents the prerelease version bump step for maintainers.
0.1.0-beta.1
Documentation and release-hardening beta.
- Aligns docs with the scoped npm package, command surface, config defaults, and strict Markdown step grammar.
- Documents runtime profiles, browser aliases, recorder artifact paths, visual checkpoint config, and public API exports.
- Adds a dedicated docs changelog page and links it from the main docs surface.
- Verifies the published
@asserthive/webtest-ai@betapackage with live registry and tarball smoke checks.