Testing with Kin Browser
Kin Browser runs checks written as plain steps — the same steps an agent takes — in a headless browser of its own: on your laptop while you build, or in CI with no window at all. No selectors and no scripts: steps find things the way a person does, by what they are called.
kin-browser test # every *.kin.yaml under ./kin-tests
kin-browser test checkout.kin.yaml --browser chromium,firefox,webkit
kin-browser test --repeat 10 # find unstable (flaky) steps
kin-browser ci # CI defaults: 1 retry, traces of failures, JUnit, a PR/MR commentResults go to kin-test-results/: report.html (open it in a browser),
summary.md, junit.xml (with ci or --junit), and for each failure a bug
report folder (screenshot, steps, errors) and, with --trace, a Playwright
trace (npx playwright show-trace <file>).
A test file
kin-tests/shop.kin.yaml:
name: Shop
base: http://localhost:3000 # opens are relative to it, and it may be opened (even on localhost)
device: mobile # optional: desktop (default) or mobile
locale: ms-MY # optional, with timezone: Asia/Kuala_Lumpur
browsers: [chromium, firefox] # optional; --browser overrides
state: shopper # optional: a saved sign-in every test starts with (save-state)
before: # steps before every test
- open: /
tests:
- name: search finds a MacBook
steps:
- open: /search
- fill: { Search: MacBook }
- press: Enter
- expect: MacBook Pro # text shown
- expect: { role: button, name: Add to cart, state: enabled }
- click: Add to cart # a button or link, by its name
- expect: { text: "Cart (1)", timeout: 10000 }
- noErrors: true # no script errors on the page
- noFailedRequests: true # no 4xx/5xx
- name: sign-up sends a code
steps:
- open: /signup
- fill: { Name: "{{fake.name}}", Email: "{{fake.email}}", Phone: "{{fake.phone}}" }
- click: Create account
- inbox: { wait: 60 } # the email arrived (start with --inbox)
- name: looks right and is accessible
steps:
- open: /
- visual: home # compared with .kin-baselines/home.png
- a11y: { max: { critical: 0, serious: 2 } }
- perf: { lcp: 2500, cls: 0.1 }Steps
| Step | Does |
|---|---|
open: /path or a full URL | Opens it and waits for the page to settle |
click: Name or { role: link, name: Name } | Clicks the button or link with that name |
hover: Name | Points at it |
fill: { Label: value } | Fills fields by their labels; true/false for checkboxes, an option for dropdowns. {{fake.email}}, {{fake.name}}, {{fake.phone}}, {{fake.mykad}}, {{fake.postcode}}, {{fake.cardNumber}}… fill one consistent made-up person |
press: Enter | A key or chord |
scroll: down, up or end | Scrolls; end until nothing more loads |
back: true | Goes back |
wait: "text" or wait: 2000 | Waits for text, or milliseconds |
expect: "text" | The text is shown ({ text, gone: true } for gone) |
expect: { role, name, state } | state: visible, hidden, enabled, disabled, checked, unchecked; or value: "…" |
expect: { count: 3, role: listitem } | How many match |
expectUrl: /thanks / expectTitle: Thank you | The address or title contains it |
noErrors: true / noFailedRequests: true | No script errors / no 4xx-5xx responses so far |
visual: name or { name, threshold: 0.5, ignore: [Updated at] } | Pixel comparison with a baseline (saved the first time) |
a11y: { max: { critical: 0 } } | Accessibility problems at most these many |
perf: { lcp: 2500, cls: 0.1, load: 4000, kb: 3000 } | Speed budgets |
request: { url: /api/health, status: 200 } | An API call with the browser's sign-in |
mock: { url: /api/items, status: 500, json: { error: down } } | Answer the page's requests with test data |
conditions: offline (or slow-3g, fast-3g, none) | Network conditions |
loadState: name / saveState: name | Sign-ins |
recipe: { name: check-invoice, params: { invoice_no: INV-1 } } | A saved recipe |
inbox: { to: [email protected], wait: 60 } | An email arrived in the test inbox |
screenshot: name | Saves a picture |
Each test starts signed out (unless isolate: false), with no tabs, a normal
network and no mocks. A test fails at its first failing step and the run moves
to the next. --retries 2 tries a failing test again; one that passes on a
retry is flaky. --repeat 10 runs everything ten times and lists the
steps that failed sometimes, and why (timing, element, network, visual).
Baselines
visual keeps its pictures in .kin-baselines/ next to the test file. Commit
them. A failing comparison leaves <name>.diff.png (the changes in red) and
<name>.actual.png beside the baseline. When the change is intended, accept
it: kin-browser test --update-baselines --grep "looks right".
Sign-ins
Save a sign-in once, use it in every test:
kin-browser start
kin-browser open https://staging.example.com/login # sign in there yourself
kin-browser save-state staging-adminthen state: staging-admin in the test file. Only cookies are restored; a site
that keeps its sign-in in page storage needs to be signed into in the test.
Other browsers
kin-browser install-browsers firefox webkit once (add --with-deps on a
fresh Linux machine), then --browser chromium,firefox,webkit. WebKit is
Safari's engine.
Test inbox
kin-browser start --inbox 2525 listens for email on 127.0.0.1:2525: point
your app's SMTP settings there in development (no username or password
needed). Agents read it with inbox; tests with the inbox step. Or use the
Mailpit your team runs: --inbox http://localhost:8025.
In CI
kin-browser ci exits 1 when a test fails (and, with --fail-on-flaky, when
one is flaky), writes JUnit to kin-test-results/junit.xml, and posts a
summary on the pull request (GitHub, with GITHUB_TOKEN) or merge request
(GitLab, with GITLAB_TOKEN). --no-comment turns that off.
GitHub Actions
name: UI tests
on: [pull_request]
permissions: { contents: read, pull-requests: write }
jobs:
kin:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with: { node-version: 22 }
- run: npm ci && npm run build && (npm start &) && npx wait-on http://localhost:3000
- name: Kin Browser
run: |
curl -fsSL https://kinbrowser.com/download/latest/kin-browser-linux.tar.gz | tar xz
./kin-browser-linux/kin-browser install-browsers chromium --with-deps
./kin-browser-linux/kin-browser ci kin-tests/
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- uses: actions/upload-artifact@v4
if: always()
with: { name: kin-test-results, path: kin-test-results }
- uses: mikepenz/action-junit-report@v4
if: always()
with: { report_paths: kin-test-results/junit.xml }GitLab CI
kin-browser:
image: node:22-bookworm
script:
- npm ci && npm run build && (npm start &) && npx wait-on http://localhost:3000
- curl -fsSL https://kinbrowser.com/download/latest/kin-browser-linux.tar.gz | tar xz
- ./kin-browser-linux/kin-browser install-browsers chromium --with-deps
- ./kin-browser-linux/kin-browser ci kin-tests/
variables:
GITLAB_TOKEN: $KIN_GITLAB_TOKEN # a project access token with api scope, for the MR comment
artifacts:
when: always
paths: [kin-test-results]
reports:
junit: kin-test-results/junit.xmlDocker
packaging/ci/Dockerfile builds an image with Chromium, Firefox and WebKit in it:
docker build -f packaging/ci/Dockerfile -t kin-browser-ci .
docker run --rm -v "$PWD:/work" -w /work --network host kin-browser-ci ci kin-tests/From a recording to a test
Record a task in the app (or let an agent do it with recording on), then:
kin-browser export-test last --language typescript --out tests/checkout.spec.tsor Export as a Playwright test in the Recordings player. The test finds
elements by role and name, as Kin Browser did; hidden text (passwords, codes)
becomes KIN_SECRET_1… environment variables, and a recipe's parameters
become KIN_<NAME> variables.
Bug reports
kin-browser report "Checkout button does nothing" writes the steps taken,
the page's errors and failed requests, a screenshot and the environment to a
folder (Markdown and HTML). File it straight away:
GITHUB_TOKEN=… kin-browser report "Checkout broken" --to github:acme/shop --labels bug,qa
GITLAB_TOKEN=… kin-browser report "Checkout broken" --to gitlab:acme/shop
JIRA_URL=https://acme.atlassian.net JIRA_EMAIL=… JIRA_TOKEN=… kin-browser report "Checkout broken" --to jira:SHOPThe team's library
Recipes and site notes, shared:
kin-browser library sync ./kin-library # a folder in your repo: commit kin-library.json
kin-browser library pull https://agent.kinbrowser.com --key kbc_…
kin-browser library push https://agent.kinbrowser.com --key kbc_…The newer recipe wins; the one it replaces is kept in that machine's history.
Screen sizes
kin-browser sweep http://localhost:3000/checkout --devices all # 11 sizes, problems by element
kin-browser sweep http://localhost:3000 --sizes 1280x800,1600x1000 --theme both
kin-browser screens --current # your tab as it is now, every size, PNGNamed devices: phone-small (375×667), phone (390×844), phone-large
(430×932), android (412×915), tablet (820×1180), tablet-landscape
(1180×820), ipad-pro (1024×1366), laptop (1366×768), laptop-hd
(1440×900), desktop (1920×1080), wide (2560×1440), or all.
Each run saves its pictures and an index.html contact sheet under
sweeps/<time>/, and reports, per size:
- wider than the screen, with the elements that stick out and how far;
- text under 12px;
- tap targets under 24px (phones and tablets only; links inside a sentence are exempt, as WCAG allows).
The pictures are CSS pixels (1×). For a user manual, --png (the default for
screens) keeps text sharp; --full takes the whole page.
New in 1.3: tests written for you, API contracts, load, fixes
A test from words
kin-browser write-test "Sign up with a new email, confirm the welcome message, then sign out" --url http://localhost:3000/signupClaude walks the flow in a headless browser, checking each result as it goes.
The walk becomes a kin.yaml test: steps by name (click: Create account,
fill: { Email: … }), never by ref, with the checks it made as expect
steps. The test is run. If it fails, Claude reads the failure, fixes the file
and runs it again (--fixes, 2 by default). The command waits, then writes
kin-tests/<name>.kin.yaml. Keep it and edit it like any test. It needs a
Claude key (kin-browser ai key), and its cost shows in kin-browser tasks.
API contract checks
kin-browser api-contract record orders http://localhost:3000/orders # on the release you trust
kin-browser api-contract check orders http://localhost:3000/orders # on the next onerecord keeps the JSON API calls the page made: each endpoint (with ids
folded into :id), the shape of its answer (fields and their types, through
arrays) and how long it took. check opens the page again and compares:
- Breaking: a field removed, a type changed (
qty: was number, now string), or an endpoint that now answers 4xx/5xx. - Slower: more than 1.5× and 300 ms slower than before.
- New: endpoints and fields that were not there before (not a failure).
It exits 1 on a breaking change, so it can gate a release in CI.
Load-lite
kin-browser load kin-tests/checkout.kin.yaml --users 20 --rounds 2
kin-browser load --url http://localhost:3000 --users 10One journey, run by up to 20 headless browsers at once. You get the whole
journey's p50, p95 and maximum time, the slowest steps, and the failures.
Real browsers find what a request-level load test misses (slow scripts,
render-blocking calls, sessions colliding), at a scale a laptop can run. It
runs only against your own app: an address Kin Browser was started with
--allow-local, or a host the team policy lists under load.hosts.
Accessibility fixes
kin-browser a11y-fix http://localhost:3000 # each problem with its fix
kin-browser a11y-fix --comment --out a11y.md # as a PR commentFor each problem a11y finds, the element as rendered and its fix:
alt="…" from the image's name, aria-label from an icon or the link's
address, a field's label from its placeholder or name, lang, the heading
level to use, and for low contrast a CSS rule with the nearest colour that
reaches 4.5:1 (3:1 for large text) against its background. Each comes with
a selector to find it in your source.
New in 1.4: every pull request, triage, regions and replay
On every pull request
kin-browser ci init github # writes .github/workflows/kin-browser.yml
kin-browser ci init gitlab # writes .gitlab-ci.kin-browser.yml to includekin-browser ci runs the kin.yaml tests. It also runs any of these checks
you name, and puts all the results in one comment on the pull or merge
request:
kin-browser ci kin-tests --allow-local localhost:3000 \
--contract orders=http://localhost:3000/orders,cart=http://localhost:3000/cart \
--a11y http://localhost:3000/ \
--score http://localhost:3000/,http://localhost:3000/pricing- API contracts: the first run records each one in
kin-tests/.kin-contracts/(commit it); later runs check against it. A breaking change fails the build. - Accessibility: each problem with its fix, as in
a11y-fix --comment. - Scores: performance, accessibility, SEO and best practices per page, with the change since the last run.
It also sets a kin-browser status on the commit: GitHub's statuses (with
GITHUB_TOKEN) or GitLab's (with GITLAB_TOKEN). Make that status required,
and nothing merges red. --no-comment and --no-status turn either off.
Is it a regression? Visual triage
kin-browser triage-visual home [--accept-intended]After visual fails, triage-visual boxes the changed areas. A change of a
few pixels is noise (anti-aliasing, a sub-pixel shift), decided without
the model. Anything bigger goes to Claude, with the baseline, the new picture
and the diff of the largest areas, and comes back intended (a deliberate
change that looks finished), regression (cut-off text, a missing element,
a broken layout) or noise, with one sentence of why. --accept-intended
makes an intended change the new baseline. Otherwise accepting stays a
person's visual --update.
The same journey from every region
kin-browser region-check checkout https://shop.example.my/checkout --expect "Pay now"The page is opened (and the text checked) from this machine and from each
fleet machine at once: one per country, with fleet add … --region SG. You get up or down and how long, per region. Each check is
kept, and the Work page's Regions shows a map with each region's uptime
and p50/p95 time over 7 days. Run it every 5 to 15 minutes with an automation
to monitor.
Session replay in bug reports
kin-browser replay start
# … do the steps …
kin-browser report "Checkout button does nothing"While a replay is on, Kin Browser keeps a picture after each step (up to 120)
and every request with its status and time. report then includes
replay/replay.html: a slider over the steps, showing the page, the step in
words, the requests and the errors up to that moment. replay stop writes
one on its own.
New in 1.5: exploring, healing, budgets, browsers and mocks
An exploratory tester
kin-browser explore http://localhost:3000 --focus "checkout" --minutes 20Claude uses the app the way people do and the way they get it wrong: empty,
long, emoji, Malay and Chinese input; back and forward; reloading
mid-flow. It checks logs after each try. Each bug it actually saw becomes a
bug report with a replay. The path it walked comes back as a kin.yaml test to
keep.
Test healing
When a step fails because a button or link was renamed, the failure says what
the page calls it now: nothing called “Add to cart” (did you mean “Add to bag”?). kin-browser ci writes fixed copies of those test files to
kin-test-results/healed/ and lists each fix in the PR comment, for a
person to keep.
Performance budgets per route
# kin-tests/perf-budgets.yaml
base: http://localhost:3000
routes:
/: { lcp: 2500, cls: 0.1, kb: 1500 }
/checkout: { lcp: 3000, kb: 2000, requests: 80 }kin-browser ci kin-tests --budgets kin-tests/perf-budgets.yamlEach route is measured and compared with its budget, with the change since
the last run (kept in .kin-contracts/perf-history.jsonl). A route over
budget fails the build and shows in red in the PR comment.
The same page in every browser
kin-browser install-browsers firefox webkit
kin-browser browser-matrix https://staging.example.my --devices phone,tablet,laptopFirefox and WebKit (Safari) are each compared with Chromium at every size, with a diff picture for each pair. This shows where a page renders differently in another browser, rather than in another release.
Mocks from API contracts
kin-browser mock-contract orders # in the page (start with --allow-mock)
kin-browser mock-server orders --port 4010 # for the app's own testsA recorded API contract (api-contract record) now keeps one real answer per
endpoint. mock-contract makes the page get those answers instead of the
backend's. mock-server serves them on a port, so front-end tests run with
no backend at all. Contracts recorded before 1.5 need recording again.