Users ask Claude, ChatGPT or Gemini to install, upgrade or migrate your product. The agent works from what it remembers and from your docs. Your page analytics show the visits, not whether the job got done.
The first report is free. The price for the next five jobs is printed on it.
A short page in your docs that takes an agent through one job, start to finish: the steps in order, and the facts each step depends on.
For example: “Move our data to the new major release in a one-hour window.” The fact that decides step three may sit in the release notes, not on the page the agent reads.
Agents run it before it ships, and again on every release.
A person reads the concepts, compares the options and comes back next week. An agent often takes the first page that looks right and turns it into commands in one pass. It mixes up versions, follows links to pages that moved, and misses the fact that sits in the release notes.
Sometimes it never opens your docs at all and answers from memory, for an older release.
When the plan fails, your user doesn’t see a docs problem. They see a product that doesn’t work with their AI, and they try the next one.
Nobody on your side sees this. Page views look normal. Nothing in them says whether the job worked.
agent attempts that skipped Valkey’s migration guide first tried to forge Redis’s binary data format. The ones that read it went straight to a key-by-key copy.
Can Agents Run Valkey? · 5 models · 105 graded jobs
We don’t do SEO or “AI visibility”. A page can rank first and still lead an agent into a broken plan. We measure what happens after your user asks.
We ask with no web search: which release, which install path, which commands. Models learn from whatever was online when they were trained, old releases included.
A job that names no product, like “add a cache to this app”. We record what the agent installs, not what it mentions.
A real install and a real job, with web search or with your docs. Graded on the end state.
The same job from memory, with web search, and with your docs. Claude, ChatGPT, Gemini and open-source models, at least three times each.
On a real install, we check what the agent left behind. When we check written plans, the checks are fixed before anyone reads an answer.
For the step that breaks: a runbook for the job, or an edit to the page that misled the agent. Ready text for your writers to review.
Yours, your docs’ and the models’. A new model version can repeat old mistakes and add new ones.
Your guides explain how the product works, for people who want to understand it. A runbook serves one job, for a reader that has to act now.
It also carries what the job needs and the guide leaves out: the fact from the release notes, the check before a step that can’t be undone, the trap we saw agents fall into.
Where agents find it: a page linked from your guide and from llms.txt, or the same text through an MCP server or an agent skill.
| A docs page | A runbook | |
|---|---|---|
| Reader | A person learning the product | An agent doing one job |
| Scope | A feature or a concept | One job, start to finish |
| Shape | Concepts, options, reference | Steps in order, with the facts each step needs |
| Checked by | Review | Agents, before it ships and on every release |
| Lives in | Your docs | Your docs, next to the guide |
We write the runbook from your docs, release notes and our runs. Your writers review it like any pull request.
A suggested change to an existing page links to the agent run where it went wrong.
We work from your public docs and a test install, the way a new user would.
ChatGPT-User, Claude-User, Perplexity-User.utm_source=chatgpt.com.Our runs show the rest: every search, every page the agent opened, where it hit a 404 or a redirect, and what it did next.
Every prompt word for word, every run with its full log, and the checks we fixed before reading any answer. When our own rig fails, we say so.
Agents needed a way to check a new API key. One agent in 11 looked for a “who am I” endpoint and got a 404. Headless Domains shipped the endpoint and one sentence telling agents to use it.
Agents had to sign up and get a working API key. We added five lines of directions: where the agent entrance is, which call to make, how to check the key.
The guide says there is no supported path from Redis 7.4 and later, and stops there. Agents that read it went straight to a key-by-key copy. The missing step is a documented copy procedure.
We publish our measurements in a series called Can Agents Run It?: Supabase, Valkey and PostgreSQL. The Valkey test kit is open source on GitHub, so you can run it yourself.
From memory, with search and with your docs: what they knew, which pages they read, where they turned the wrong way, and whether the job finished.
For the step that breaks. Yours to publish in your docs.
The same agents on the same job after the fix, so you can see it finish.
The first report covers one job, and it’s free. The price for the next five jobs is printed on the report. After the fix, we re-check the job on every release: yours, your docs’ and the models’.
Moving to a new docs platform or structure? We run your key jobs before the switch and again after the new docs go live, so you can see what changed for agents.
No. We don’t work on getting you mentioned. We measure what agents do once a user asks: what they remember, what they pick and whether the job works.
Our lab publishes its prompts and every run, and the Valkey test kit is open source, so you can check our numbers yourself. What we sell is running the tests on your jobs, with fixes, on every release.
You can, and our public test kit is a place to start. It takes several agents, a check for every job and a re-run after every release. Most teams never build that rig. We already run it.
Not reliably. Models learn from what was online when they were trained, old pages included. A new version can repeat old mistakes and add new ones, so the check runs again after every model release.
Claude, ChatGPT and Gemini, plus the open-source models developers run themselves. We’ve worked with 25+ models so far, so we can match the mix your users bring.
One job that matters and a way to install your product: open source, a free tier or a trial. We don’t install anything on your side.
Tell us about your product and one job your users hand to AI agents. We’ll run it and send you the report.
Prefer to talk first? Book a call →
We’ll reply by email to agree on the job and the release to test.
No reply? Write to eugene@docka.ai.