The First Useful AI Agents May Be QA Testers

Every few weeks, someone posts a demo of an AI agent booking a flight, and every few weeks, someone else posts the failure case. The agent picks the wrong date, or the wrong airport, or confidently checks out with a fare the user never approved.

Browser agents get stuck in loops clicking the same cookie banner. Shopping agents add the wrong size to the cart. Personal assistant agents send calendar invites for the wrong week.

The pattern repeats often enough that “AI agents don’t work yet” has become a reasonable default position.

Except one category of agent is already doing real, unglamorous work in production right now, and almost nobody talks about it. It’s testing software. While the flashy agents fumble your travel plans, QA agents are clicking through checkout flows, resetting passwords, and filing bug reports at three in the morning.

I write about agents on Medium, and this is the one deployment pattern where the complaints I hear are about coverage, not competence.

Why Most AI Agents Still Fail in The Real World

The common failure modes of agents aren’t random. They cluster around a few structural problems, and once you see them, you can predict which agent products will disappoint before you ever try them.

• Ambiguous goals: “Book me a good flight to Tokyo” hides a dozen unstated preferences. Direct or one stop? Morning or red-eye? Which airline alliance? A human assistant asks follow-up questions. Most agents guess, and a guess that’s 90% right on each of five decisions is wrong more often than not overall.
• Unbounded action spaces: A browser agent on the open web can go anywhere and do anything. That freedom is exactly the problem. The space of possible actions is so large that the agent spends most of its reasoning budget just deciding what not to do, and small perception errors compound into loops or dead ends.
• Real consequences for mistakes: When a booking agent buys the wrong ticket, someone pays for it. When an email agent sends a half-finished draft to a client, someone apologizes for it. Every error lands on a person, which means the tolerance for error is close to zero, and no current model is close to zero.
• No clear definition of done: How does a research agent know its report is complete? How does a shopping agent know it found the best deal? Without a verifiable end state, agents either stop too early or burn tokens rewriting something nobody asked for.

Any agent product that inherits all four of these problems is fighting uphill. The interesting question is what happens when a task has none of them.

Why QA Work is Different

Software testing is that task, and it’s almost eerie how cleanly it inverts each failure mode. The environment is bounded. A QA agent operates inside one application, on a staging URL or preview deploy that the team controls.

It doesn’t need to understand the whole web. It needs to understand your signup form, your checkout, your dashboard.

A finite surface with finite states is exactly the setting where current models behave well.

Failure is cheap. If a test agent misclicks in staging, nothing happens. No money moves, no customer sees it, no one apologizes.

The agent can retry, explore a weird path, or crash entirely, and the cost is a few minutes of compute. Compare that to a booking agent, where a single mistake costs real money and trust.

Verification is built in. A test has a definition of done that most agent tasks can only dream of: the user either ends up logged in or doesn’t, the order confirmation either appears or doesn’t. The agent checks an outcome against an expectation, which means it can grade its own work.

And the work itself is human-hostile. Nobody’s career goal is to manually re-test the password reset flow for the 40th time before a release. Regression testing is repetitive, boring, and endless, which is why it gets skipped under deadline pressure.

Handing it to an agent doesn’t threaten anyone’s favorite part of the job. It removes the part everyone was already avoiding.

Put those four properties together and QA stops looking like a niche agent use case. It looks like the use case the current generation of models was accidentally built for.

QA.tech in Practice

QA.tech is one of the clearer examples of what this looks like as a product. The category is agentic QA: testing where an AI agent works toward a user goal instead of executing a script.

You describe a test as a plain-language goal, something like “sign up with a new account, confirm the email, and complete onboarding,” and a QA agent executes it in a real browser like a real user, with a goal in mind, then reports what happened with screenshots and video.

Creating a test suite in QA.tech

Because the tests are goals rather than scripts, they survive UI changes. Suites built on legacy tools like Playwright, Cypress, or Selenium break when a button gets renamed because they depend on selectors and exact steps. A QA agent re-navigates around the change the way a human tester would, which kills the single biggest maintenance tax in test automation.

The company’s use case library covers 36 scenarios, and a few stand out for how far past scripted automation they go:

• Authentication flows including the parts traditional tools can’t touch, like opening an email inbox or reading an SMS code to complete a login
• Multi-user workflows where two agents run as two different users in isolated sessions, so one can submit a request and the other can approve it
• Regression plans where a 50-test suite that took hours by hand runs in parallel and finishes in around ten minutes
• Multi-environment testing where the same test plan points at local, staging, production, or a preview URL, and QA.tech handles the environment URLs, auth layers, and firewall rules

The pull request integration is where this becomes a daily workflow rather than a tool you remember to run. A GitHub or GitLab bot watches the repo, detects the preview deployment when a PR opens, runs exploratory and regression tests against it, and posts pass/fail results back to the PR thread before anyone merges.

QA.tech pull request integration example

The PR comment is the surface, not the product. Full run details, screenshots, video, and history live in the QA.tech dashboard, so the team can dig into any failure beyond the thread. Pricer, a retail tech company, runs this on every pull request and reports saving 390 hours of testing per quarter.

The piece I find most interesting as someone who ships code with Claude Code is the MCP integration. Connect QA.tech as an MCP server and your coding agent, whether that’s Claude Code, OpenAI Codex, Cursor, or Continue, can query your test suite, trigger runs, and pull back results without leaving the coding environment. A qatech init command even generates Claude Code subagent and skill files directly in your repo.

This moves the feedback earlier than the PR bot does. Instead of push, wait for the preview deploy, then read results, the loop becomes code, trigger a run from inside the editor, fix, then push. The two approaches complement each other: MCP is the tight inner loop for the developer during active coding, and the PR check is the automated gate that runs without any manual trigger and gives the whole team visibility.

QA as the Training Ground

There’s a bigger pattern here than one product category, and I think it explains where agents go next.

Testing is where agents learn to use software before anyone trusts them to use it on your behalf. Every test run is an agent navigating a real web or mobile app: reading a page, forming an intent, clicking, typing, checking whether the result matches the expectation. That is the exact skill set a booking agent or an operations agent needs.

The difference is that in QA, the stakes are inverted. The agent practices on software where mistakes are the product, not the problem.

The history of automation follows this shape. Autonomous driving didn’t start with taxis, it started with highway lane-keeping, the bounded version of the problem. Robots didn’t start in homes, they started in factory cages.

Agents are following the same path: master the constrained environment first, earn trust through verified output, then expand outward.

So when a QA agent runs your checkout flow ten thousand times across releases, it isn’t just protecting your revenue. It’s generating exactly the kind of grounded, verified interaction data that the next generation of general-purpose agents will be built on. The companies running agents in testing today are collecting the receipts that consumer agents don’t have yet.

What this Means for Tech and Product Teams