kinbrowserdocs

Testing with Kin Browser

Kin Browser runs checks written as plain steps — the same steps an agent takes — in a headless browser of its own: on your laptop while you build, or in CI with no window at all. No selectors and no scripts: steps find things the way a person does, by what they are called.

kin-browser test                      # every *.kin.yaml under ./kin-tests
kin-browser test checkout.kin.yaml --browser chromium,firefox,webkit
kin-browser test --repeat 10          # find unstable (flaky) steps
kin-browser ci                        # CI defaults: 1 retry, traces of failures, JUnit, a PR/MR comment

Results go to kin-test-results/: report.html (open it in a browser), summary.md, junit.xml (with ci or --junit), and for each failure a bug report folder (screenshot, steps, errors) and, with --trace, a Playwright trace (npx playwright show-trace <file>).

A test file

kin-tests/shop.kin.yaml:

name: Shop
base: http://localhost:3000     # opens are relative to it, and it may be opened (even on localhost)
device: mobile                  # optional: desktop (default) or mobile
locale: ms-MY                   # optional, with timezone: Asia/Kuala_Lumpur
browsers: [chromium, firefox]   # optional; --browser overrides
state: shopper                  # optional: a saved sign-in every test starts with (save-state)

before:                          # steps before every test
  - open: /

tests:
  - name: search finds a MacBook
    steps:
      - open: /search
      - fill: { Search: MacBook }
      - press: Enter
      - expect: MacBook Pro                          # text shown
      - expect: { role: button, name: Add to cart, state: enabled }
      - click: Add to cart                           # a button or link, by its name
      - expect: { text: "Cart (1)", timeout: 10000 }
      - noErrors: true                               # no script errors on the page
      - noFailedRequests: true                       # no 4xx/5xx

  - name: sign-up sends a code
    steps:
      - open: /signup
      - fill: { Name: "{{fake.name}}", Email: "{{fake.email}}", Phone: "{{fake.phone}}" }
      - click: Create account
      - inbox: { wait: 60 }                          # the email arrived (start with --inbox)

  - name: looks right and is accessible
    steps:
      - open: /
      - visual: home                                 # compared with .kin-baselines/home.png
      - a11y: { max: { critical: 0, serious: 2 } }
      - perf: { lcp: 2500, cls: 0.1 }

Steps

StepDoes
open: /path or a full URLOpens it and waits for the page to settle
click: Name or { role: link, name: Name }Clicks the button or link with that name
hover: NamePoints at it
fill: { Label: value }Fills fields by their labels; true/false for checkboxes, an option for dropdowns. {{fake.email}}, {{fake.name}}, {{fake.phone}}, {{fake.mykad}}, {{fake.postcode}}, {{fake.cardNumber}}… fill one consistent made-up person
press: EnterA key or chord
scroll: down, up or endScrolls; end until nothing more loads
back: trueGoes back
wait: "text" or wait: 2000Waits for text, or milliseconds
expect: "text"The text is shown ({ text, gone: true } for gone)
expect: { role, name, state }state: visible, hidden, enabled, disabled, checked, unchecked; or value: "…"
expect: { count: 3, role: listitem }How many match
expectUrl: /thanks / expectTitle: Thank youThe address or title contains it
noErrors: true / noFailedRequests: trueNo script errors / no 4xx-5xx responses so far
visual: name or { name, threshold: 0.5, ignore: [Updated at] }Pixel comparison with a baseline (saved the first time)
a11y: { max: { critical: 0 } }Accessibility problems at most these many
perf: { lcp: 2500, cls: 0.1, load: 4000, kb: 3000 }Speed budgets
request: { url: /api/health, status: 200 }An API call with the browser's sign-in
mock: { url: /api/items, status: 500, json: { error: down } }Answer the page's requests with test data
conditions: offline (or slow-3g, fast-3g, none)Network conditions
loadState: name / saveState: nameSign-ins
recipe: { name: check-invoice, params: { invoice_no: INV-1 } }A saved recipe
inbox: { to: [email protected], wait: 60 }An email arrived in the test inbox
screenshot: nameSaves a picture

Each test starts signed out (unless isolate: false), with no tabs, a normal network and no mocks. A test fails at its first failing step and the run moves to the next. --retries 2 tries a failing test again; one that passes on a retry is flaky. --repeat 10 runs everything ten times and lists the steps that failed sometimes, and why (timing, element, network, visual).

Baselines

visual keeps its pictures in .kin-baselines/ next to the test file. Commit them. A failing comparison leaves <name>.diff.png (the changes in red) and <name>.actual.png beside the baseline. When the change is intended, accept it: kin-browser test --update-baselines --grep "looks right".

Sign-ins

Save a sign-in once, use it in every test:

kin-browser start
kin-browser open https://staging.example.com/login     # sign in there yourself
kin-browser save-state staging-admin

then state: staging-admin in the test file. Only cookies are restored; a site that keeps its sign-in in page storage needs to be signed into in the test.

Other browsers

kin-browser install-browsers firefox webkit once (add --with-deps on a fresh Linux machine), then --browser chromium,firefox,webkit. WebKit is Safari's engine.

Test inbox

kin-browser start --inbox 2525 listens for email on 127.0.0.1:2525: point your app's SMTP settings there in development (no username or password needed). Agents read it with inbox; tests with the inbox step. Or use the Mailpit your team runs: --inbox http://localhost:8025.

In CI

kin-browser ci exits 1 when a test fails (and, with --fail-on-flaky, when one is flaky), writes JUnit to kin-test-results/junit.xml, and posts a summary on the pull request (GitHub, with GITHUB_TOKEN) or merge request (GitLab, with GITLAB_TOKEN). --no-comment turns that off.

GitHub Actions

name: UI tests
on: [pull_request]
permissions: { contents: read, pull-requests: write }
jobs:
  kin:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 22 }
      - run: npm ci && npm run build && (npm start &) && npx wait-on http://localhost:3000
      - name: Kin Browser
        run: |
          curl -fsSL https://kinbrowser.com/download/latest/kin-browser-linux.tar.gz | tar xz
          ./kin-browser-linux/kin-browser install-browsers chromium --with-deps
          ./kin-browser-linux/kin-browser ci kin-tests/
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      - uses: actions/upload-artifact@v4
        if: always()
        with: { name: kin-test-results, path: kin-test-results }
      - uses: mikepenz/action-junit-report@v4
        if: always()
        with: { report_paths: kin-test-results/junit.xml }

GitLab CI

kin-browser:
  image: node:22-bookworm
  script:
    - npm ci && npm run build && (npm start &) && npx wait-on http://localhost:3000
    - curl -fsSL https://kinbrowser.com/download/latest/kin-browser-linux.tar.gz | tar xz
    - ./kin-browser-linux/kin-browser install-browsers chromium --with-deps
    - ./kin-browser-linux/kin-browser ci kin-tests/
  variables:
    GITLAB_TOKEN: $KIN_GITLAB_TOKEN     # a project access token with api scope, for the MR comment
  artifacts:
    when: always
    paths: [kin-test-results]
    reports:
      junit: kin-test-results/junit.xml

Docker

packaging/ci/Dockerfile builds an image with Chromium, Firefox and WebKit in it:

docker build -f packaging/ci/Dockerfile -t kin-browser-ci .
docker run --rm -v "$PWD:/work" -w /work --network host kin-browser-ci ci kin-tests/

From a recording to a test

Record a task in the app (or let an agent do it with recording on), then:

kin-browser export-test last --language typescript --out tests/checkout.spec.ts

or Export as a Playwright test in the Recordings player. The test finds elements by role and name, as Kin Browser did; hidden text (passwords, codes) becomes KIN_SECRET_1… environment variables, and a recipe's parameters become KIN_<NAME> variables.

Bug reports

kin-browser report "Checkout button does nothing" writes the steps taken, the page's errors and failed requests, a screenshot and the environment to a folder (Markdown and HTML). File it straight away:

GITHUB_TOKEN=… kin-browser report "Checkout broken" --to github:acme/shop --labels bug,qa
GITLAB_TOKEN=… kin-browser report "Checkout broken" --to gitlab:acme/shop
JIRA_URL=https://acme.atlassian.net JIRA_EMAIL=… JIRA_TOKEN=… kin-browser report "Checkout broken" --to jira:SHOP

The team's library

Recipes and site notes, shared:

kin-browser library sync ./kin-library     # a folder in your repo: commit kin-library.json
kin-browser library pull https://agent.kinbrowser.com --key kbc_…
kin-browser library push https://agent.kinbrowser.com --key kbc_…

The newer recipe wins; the one it replaces is kept in that machine's history.

Screen sizes

kin-browser sweep http://localhost:3000/checkout --devices all             # 11 sizes, problems by element
kin-browser sweep http://localhost:3000 --sizes 1280x800,1600x1000 --theme both
kin-browser screens --current                                              # your tab as it is now, every size, PNG

Named devices: phone-small (375×667), phone (390×844), phone-large (430×932), android (412×915), tablet (820×1180), tablet-landscape (1180×820), ipad-pro (1024×1366), laptop (1366×768), laptop-hd (1440×900), desktop (1920×1080), wide (2560×1440), or all.

Each run saves its pictures and an index.html contact sheet under sweeps/<time>/, and reports, per size:

  • wider than the screen, with the elements that stick out and how far;
  • text under 12px;
  • tap targets under 24px (phones and tablets only; links inside a sentence are exempt, as WCAG allows).

The pictures are CSS pixels (1×). For a user manual, --png (the default for screens) keeps text sharp; --full takes the whole page.

New in 1.3: tests written for you, API contracts, load, fixes

A test from words

kin-browser write-test "Sign up with a new email, confirm the welcome message, then sign out" --url http://localhost:3000/signup

Claude walks the flow in a headless browser, checking each result as it goes. The walk becomes a kin.yaml test: steps by name (click: Create account, fill: { Email: … }), never by ref, with the checks it made as expect steps. The test is run. If it fails, Claude reads the failure, fixes the file and runs it again (--fixes, 2 by default). The command waits, then writes kin-tests/<name>.kin.yaml. Keep it and edit it like any test. It needs a Claude key (kin-browser ai key), and its cost shows in kin-browser tasks.

API contract checks

kin-browser api-contract record orders http://localhost:3000/orders   # on the release you trust
kin-browser api-contract check orders http://localhost:3000/orders    # on the next one

record keeps the JSON API calls the page made: each endpoint (with ids folded into :id), the shape of its answer (fields and their types, through arrays) and how long it took. check opens the page again and compares:

  • Breaking: a field removed, a type changed (qty: was number, now string), or an endpoint that now answers 4xx/5xx.
  • Slower: more than 1.5× and 300 ms slower than before.
  • New: endpoints and fields that were not there before (not a failure).

It exits 1 on a breaking change, so it can gate a release in CI.

Load-lite

kin-browser load kin-tests/checkout.kin.yaml --users 20 --rounds 2
kin-browser load --url http://localhost:3000 --users 10

One journey, run by up to 20 headless browsers at once. You get the whole journey's p50, p95 and maximum time, the slowest steps, and the failures. Real browsers find what a request-level load test misses (slow scripts, render-blocking calls, sessions colliding), at a scale a laptop can run. It runs only against your own app: an address Kin Browser was started with --allow-local, or a host the team policy lists under load.hosts.

Accessibility fixes

kin-browser a11y-fix http://localhost:3000            # each problem with its fix
kin-browser a11y-fix --comment --out a11y.md          # as a PR comment

For each problem a11y finds, the element as rendered and its fix: alt="…" from the image's name, aria-label from an icon or the link's address, a field's label from its placeholder or name, lang, the heading level to use, and for low contrast a CSS rule with the nearest colour that reaches 4.5:1 (3:1 for large text) against its background. Each comes with a selector to find it in your source.

New in 1.4: every pull request, triage, regions and replay

On every pull request

kin-browser ci init github        # writes .github/workflows/kin-browser.yml
kin-browser ci init gitlab        # writes .gitlab-ci.kin-browser.yml to include

kin-browser ci runs the kin.yaml tests. It also runs any of these checks you name, and puts all the results in one comment on the pull or merge request:

kin-browser ci kin-tests --allow-local localhost:3000 \
  --contract orders=http://localhost:3000/orders,cart=http://localhost:3000/cart \
  --a11y http://localhost:3000/ \
  --score http://localhost:3000/,http://localhost:3000/pricing
  • API contracts: the first run records each one in kin-tests/.kin-contracts/ (commit it); later runs check against it. A breaking change fails the build.
  • Accessibility: each problem with its fix, as in a11y-fix --comment.
  • Scores: performance, accessibility, SEO and best practices per page, with the change since the last run.

It also sets a kin-browser status on the commit: GitHub's statuses (with GITHUB_TOKEN) or GitLab's (with GITLAB_TOKEN). Make that status required, and nothing merges red. --no-comment and --no-status turn either off.

Is it a regression? Visual triage

kin-browser triage-visual home [--accept-intended]

After visual fails, triage-visual boxes the changed areas. A change of a few pixels is noise (anti-aliasing, a sub-pixel shift), decided without the model. Anything bigger goes to Claude, with the baseline, the new picture and the diff of the largest areas, and comes back intended (a deliberate change that looks finished), regression (cut-off text, a missing element, a broken layout) or noise, with one sentence of why. --accept-intended makes an intended change the new baseline. Otherwise accepting stays a person's visual --update.

The same journey from every region

kin-browser region-check checkout https://shop.example.my/checkout --expect "Pay now"

The page is opened (and the text checked) from this machine and from each fleet machine at once: one per country, with fleet add … --region SG. You get up or down and how long, per region. Each check is kept, and the Work page's Regions shows a map with each region's uptime and p50/p95 time over 7 days. Run it every 5 to 15 minutes with an automation to monitor.

Session replay in bug reports

kin-browser replay start
# … do the steps …
kin-browser report "Checkout button does nothing"

While a replay is on, Kin Browser keeps a picture after each step (up to 120) and every request with its status and time. report then includes replay/replay.html: a slider over the steps, showing the page, the step in words, the requests and the errors up to that moment. replay stop writes one on its own.

New in 1.5: exploring, healing, budgets, browsers and mocks

An exploratory tester

kin-browser explore http://localhost:3000 --focus "checkout" --minutes 20

Claude uses the app the way people do and the way they get it wrong: empty, long, emoji, Malay and Chinese input; back and forward; reloading mid-flow. It checks logs after each try. Each bug it actually saw becomes a bug report with a replay. The path it walked comes back as a kin.yaml test to keep.

Test healing

When a step fails because a button or link was renamed, the failure says what the page calls it now: nothing called “Add to cart” (did you mean “Add to bag”?). kin-browser ci writes fixed copies of those test files to kin-test-results/healed/ and lists each fix in the PR comment, for a person to keep.

Performance budgets per route

# kin-tests/perf-budgets.yaml
base: http://localhost:3000
routes:
  /:          { lcp: 2500, cls: 0.1, kb: 1500 }
  /checkout:  { lcp: 3000, kb: 2000, requests: 80 }
kin-browser ci kin-tests --budgets kin-tests/perf-budgets.yaml

Each route is measured and compared with its budget, with the change since the last run (kept in .kin-contracts/perf-history.jsonl). A route over budget fails the build and shows in red in the PR comment.

The same page in every browser

kin-browser install-browsers firefox webkit
kin-browser browser-matrix https://staging.example.my --devices phone,tablet,laptop

Firefox and WebKit (Safari) are each compared with Chromium at every size, with a diff picture for each pair. This shows where a page renders differently in another browser, rather than in another release.

Mocks from API contracts

kin-browser mock-contract orders                     # in the page (start with --allow-mock)
kin-browser mock-server orders --port 4010           # for the app's own tests

A recorded API contract (api-contract record) now keeps one real answer per endpoint. mock-contract makes the page get those answers instead of the backend's. mock-server serves them on a port, so front-end tests run with no backend at all. Contracts recorded before 1.5 need recording again.