Agent-driven testing: collect no-context agent reports on fixture3 and its docs

[tartakovsky]

What

Hand fixture3 to AI agents with zero prior context about the tool. Ask them to use it on a real project and report:

• Anything ambiguous in fixture3 --help
• Anything ambiguous in error messages
• Anywhere the agent chose the wrong command or flag because the docs were misleading
• Anywhere the agent needed to read source code to figure out what was happening

Why

If fixture3 is built for agent-managed codebases, agents must be able to onboard themselves from --help alone. Every confusion is a doc gap. Every wrong choice is a UX gap.

Suggested protocol

1. Pick a real project the agent has worked on before.
2. Give the agent the URL of fixture3 and nothing else. No prior context.
3. Ask: "Install this tool and use it to add approval testing to this project. Report any docs that confused you and any command that did something other than what its help text suggested."
4. Record the full session.
5. Repeat with 3-5 different agent runs across different project shapes.

Done

A document collecting agent reports. Each ambiguity becomes a follow-up issue (help-text rewrite, error-message rewrite, command renaming, manifest field clarification).