shankardas-lt/playwright-agentic-automation-harness

An AI-assisted, **application-agnostic** end-to-end testing harness. You describe what you want in plain English inside Claude Code (for example *"write automation for the login page"*). The harness then:
1. explores the site with the Playwright MCP browser,
2. documents what it actually observed,
3. writes Page-Object-based Playwright tests,
4. runs them and generates an Allure report,
5. repairs broken locators in the source code, under hard guardrails,
6. reports only what the test results prove.
The harness is built so it never fakes a pass. Wrong prices, wrong messages and missing features stay failed and are reported. Only locator drift (the page still has the element, under a new name or markup) is repaired, and a repair is kept only after a real Playwright rerun passes with the original assertions unchanged.
- **Default target:** https://qa-demo-website.vercel.app. To test any other site you are authorized to test, change BASE_URL and ALLOWED_HOSTS in .env; the code does not need to change.
- **Stack:** Node.js ≥ 20, JavaScript ES modules, @playwright/test 1.64.0 (pinned), allure-playwright 3.13.0 + allure 3.19.1 CLI (pinned, Node-based, no Java), Playwright MCP, Claude Code agents, skills and hooks.
> **Current state (2026-10-14):** the **Login** and **Signup** pages of the default target are explored, documented in docs/ and automated:
>
> | Spec | Test cases | Page Objects |
> |---|---|---|
> | tests/specs/login.spec.js | TC-AUTH-LOGIN-001 … 011 | login-page.js, header-component.js, products-page.js |
> | tests/specs/signup.spec.js | TC-AUTH-SIGNUP-001 … 011 | signup-page.js, home-page.js, header-component.js |
>
> Latest run: 21 passed · 0 failed · 1 skipped (headless and headed; the full suite was also stable over 5 repeats). The skip is TC-AUTH-SIGNUP-003, which needs ALLOW_ACCOUNT_CREATION=true (see Environment variables). TC-AUTH-LOGIN-008 used to fail intermittently because it clicked before the app had hydrated; BasePage.goto() now waits for hydration (tests/support/app-ready.js). The rest of the site (catalog, cart, orders, profile, checkout) is **not explored** yet.
---
Quick start
Opening the reports
To open it:
- **Allure:** npm run report:allure:open
- **Playwright HTML:** npm run report:open
Environment variables
.env is git-ignored; never commit it. .env.example lists every setting.
| Variable | Purpose |
|---|---|
| BASE_URL, ALLOWED_HOSTS | Target site and the hosts the harness may touch |
| TEST_USER_EMAIL, TEST_USER_PASSWORD | Existing test account. Needed by TC-AUTH-LOGIN-002, 004, 011 and TC-AUTH-SIGNUP-004; those tests are skipped with a reason when empty |
| ALLOW_ACCOUNT_CREATION | true lets TC-AUTH-SIGNUP-003 create a new account on every run (email qa-harness+signup- @example.invalid). Default false |
| WORKERS, RETRIES, *_TIMEOUT_MS | Execution policy |
| RUN_RETENTION | How many artifacts/runs/ / directories are kept |
Open the folder in Claude Code. CLAUDE.md, the agents, the skills and the hooks are picked up from the project automatically. The Playwright MCP server must be available, either from your user settings or from a project .mcp.json.
---
How the system works
- **Agents decide; code enforces.** The rules that must never be broken are implemented in utils/ and are unit-tested, not just written in prompts:
- which files may be edited during healing,
- that a repair changes locators only,
- that a failed repair is restored,
- that the final status comes from a real rerun.
- **Playwright runs without the AI.** Every test is a normal Playwright test (npx playwright test works). Claude Code adds exploration, authoring and repair on top.
- **One orchestrator.** Claude Code subagents cannot start other subagents. The /qa-automation skill therefore turns the main conversation into the orchestrator. Alternatively, start Claude with claude --agent qaautomation.
---
Working with Claude Code
Type requests naturally. Prefixing them with /qa-automation makes the routing explicit, but plain requests also work because the skill is described as the entry point for QA work. Each example below shows what you type and what the system does.