Skip to content
AI Engineer Guide.

concept · coding · intermediate · concept 16 of 30

UI automation

0 of 30 complete

Why it matters

Some defects only exist in the assembled product: a route that never renders, a form that loses input, an auth flow that loops. Unit tests cannot see them because they stop below the browser. An agent can now click, type and read a page, which makes those defects testable, but it inherits the browser's asynchrony and the application's hidden state, so a naive run produces failures that say more about timing than about the code.

Mental model

It is a crash-test dummy: a real collision with an expensive setup. The metaphor breaks because the dummy never gets logged out mid-test or meets a cookie banner it cannot dismiss, and those are the failures that actually stop the run.

How it works

Mechanism, step by stepInteractive
  1. 1Pick the smallest user path that exercises the change and assert on visible text rather than DOM structure.
  2. 2Reset state per run with a fresh profile, a seeded database and a fixed clock where possible.
  3. 3Wait on an observable condition such as a role becoming visible, never on a fixed sleep.
  4. 4Capture console errors, failed network requests and a screenshot whenever an assertion fails.
  5. 5Run against a production build served locally so bundling differences surface in the test.
  6. 6Record a trace and replay it before editing the test when a failure appears.

Step 1 of 6

Pick the smallest user path that exercises the change and assert on visible text rather than DOM structure.

Step 1/6 · step 1

Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.

What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.

Text version

The page's steps: 1. Pick the smallest user path that exercises the change and assert on visible text rather t… 2. Reset state per run with a fresh profile, a seeded database and a fixed clock where possi… 3. Wait on an observable condition such as a role becoming visible, never on a fixed sleep. 4. Capture console errors, failed network requests and a screenshot whenever an assertion fa… 5. Run against a production build served locally so bundling differences surface in the test. 6. Record a trace and replay it before editing the test when a failure appears.

  1. Pick the smallest user path that exercises the change and assert on visible text rather than DOM structure.
  2. Reset state per run with a fresh profile, a seeded database and a fixed clock where possible.
  3. Wait on an observable condition such as a role becoming visible, never on a fixed sleep.
  4. Capture console errors, failed network requests and a screenshot whenever an assertion fails.
  5. Run against a production build served locally so bundling differences surface in the test.
  6. Record a trace and replay it before editing the test when a failure appears.

Trade-offs

ChoiceGood forCostFails when
Accessibility-role selectorsStable across refactorsRequires accessible markupComponents are divs with click handlers
CSS or test-id selectorsPrecise, quick to writeBreak on styling and refactorTeam renames ids freely
Headless browserFast, CI friendlyMisses rendering and input quirksBug is visual or device-specific
Recorded trace replayExcellent debuggingStorage cost, version couplingApp changes often
Trade-off mapInteractive

Good for

Stable across refactors

Cost

Requires accessible markup

Fails when

Components are divs with click handlers

  1. Components are divs with click handlers
  2. Team renames ids freely
  3. Bug is visual or device-specific
  4. App changes often

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Accessibility-role selectors is good for Stable across refactors, costs Requires accessible markup, and fails when Components are divs with click handlers. CSS or test-id selectors is good for Precise, quick to write, costs Break on styling and refactor, and fails when Team renames ids freely. Headless browser is good for Fast, CI friendly, costs Misses rendering and input quirks, and fails when Bug is visual or device-specific. Recorded trace replay is good for Excellent debugging, costs Storage cost, version coupling, and fails when App changes often

Failure modes

  • Flaky wait. The test passes locally and fails in CI because an async fetch was never awaited. Debug: replay the trace and read the failing step's network waterfall.
  • Stale selector. The test clicks a hidden duplicate and asserts on the wrong screen. Debug: assert the locator resolves to exactly one element before acting.
  • Auth expiry. The session times out mid-run and the failure is reported as a UI bug. Debug: check the last successful navigation against the auth cookie lifetime.

In production

Track flake rate per test and quarantine anything above a threshold. Cap total run time and retain traces for failures only, since they dominate storage.

Common misconceptions

People think: End-to-end tests are the most trustworthy.

Actually: They are the most realistic and the least stable; trust comes from a small, well-isolated set.

People think: An agent that can click can test any UI.

Actually: It still needs deterministic state and durable selectors, and autonomy does not remove flakiness.

Interview angle

Frame browser tests as an integration layer with a flake budget. A strong answer covers durable selectors, state reset, trace capture and a small critical-path set, and gives an example of a timing bug that looked like a product bug. Proposing to automate everything through the UI is the weak answer.

Sources