A machine pass before a human one

Where the scenarios come from

The scenarios come from the document that says what the system is supposed to do, checked against what the screens actually show, rather than from an agent exploring the interface and guessing at intent.

Where the two disagree, that disagreement is itself a finding, and often the most valuable one: nobody was looking for it.

What runs

Agents drive the real application through those scenarios. The same paths a person would take, executed, with the network traffic and the screens recorded as they go.

The agent that writes the scenarios is never the agent that runs them. Whoever runs them cannot rewrite them: a scenario that cannot be performed is reported as blocked, with the reason, rather than quietly adjusted into something convenient.

What a finding looks like

Written to be moved into your tracker without editing: what broke, what was expected, the exact steps that produce it again, and the evidence.

Every finding also says what it is a defect of — the product, or my own expectation. Those are not the same thing, and a report that never admits the second kind should not be trusted about the first.

Where the product has no documented rule for the behaviour, the finding says so and stops. It goes to whoever owns the product as a question, not into the tracker as a bug.

What it does not do

It does not replace a tester. It will tell you where the application and the documentation disagree, and that is where most of the findings come from. What it cannot tell you is whether the documented behaviour was right in the first place. That needs somebody who knows your users.

A test environment is the default and the first thing I ask for. Runs against production are possible where the paths are read-only and the data is agreed in advance — scoped explicitly, never assumed.

What the agents may do there is agreed in writing before the first run and enforced in the harness rather than left to judgement: read and create by default, no deletion, no edits to records the run did not create, no actions against payment, messaging or any third-party service, and an account carrying only the permissions the scenarios need. Anything beyond that is a written exception, named by you.

Every run produces a record of what was not reachable and why — a blocked login, a third party with no sandbox, a flow that needs data nobody could seed. That list ships with the report.

How you check it is not lying

Every step it claims to have taken has a file behind it — a screenshot, a network recording, a response body. A step without one is not counted, and the run says so.

Each finding is reproduced a second time, in a new session on data created from scratch, before it goes into the report.

Where a check reports something absent — no results, no button, no permission — the same check is run against a case where the thing is present. Both results go into the report.

Every finding in the report has been through both checks before I see it. I read what survives. Nothing reaches you that I have not looked at.

What you get

A report waiting at the start of the session instead of an empty checklist, and a tester whose day goes on the subtle failures rather than the visible ones. The first run lands in days, not at the end of the engagement.

Alongside it: a written record of what was not covered and why. That list ships with the report, and it is how you decide what to look at yourself.