Katalon AI Test Runner Launches: What QA Teams Should Check Before Automating Manual Tests
Katalon AI Test Runner runs existing natural-language manual tests on web and mobile apps, uses a separate AI agent to verify each step, and saves evidence for review.
Katalon AI Test Runner turns existing manual cases into automated regression runs
Katalon has launched AI Test Runner in Katalon TestOps, a capability that runs existing manual test cases written in natural language against web and mobile applications. Teams can use those cases without rewriting them as scripts or building a new automation framework first.
For QA organizations with large manual regression suites, the release targets a familiar problem: the cases exist, but there is not enough time to run them before every release. Katalon AI Test Runner runs independent cases in parallel, records the outcome of each step, and attaches evidence such as screenshots and video.
How Katalon AI Test Runner works
The workflow starts in TestOps. A tester selects an individual test case, a test suite, or a manual test run, then starts the run with AI. The system reviews the test for clarity and completeness before execution. If a step fails to explain what to do or what result to expect, the runner can flag it before the run begins.
During execution, the agent follows the natural-language instructions in a cloud browser or supported mobile environment. Katalon says test cases can run in parallel, and data-driven cases can create separate sessions for individual dataset rows.
The release separates execution from verification. One AI agent performs the test, while a separate read-only AI reviews screenshots and recorded actions to decide whether each step passed or failed. When the evidence is unclear, the system can retry the step and send it for human review instead of marking it successful automatically.
Evidence matters more than an AI-generated status
Each step produces a verdict and supporting evidence. Results can include screenshots, video, execution details, the expected result, the actual result, and an AI explanation for failed steps. This addresses a central risk in agentic testing: trusting a green status without seeing how the system reached it.
Teams should treat the output as test evidence, not an unquestionable release decision. Testers still decide whether a failure points to a product defect, a test-data issue, an environment problem, or an unclear test instruction.
What QA teams should check before adopting it
- Test case quality: Manual cases need explicit actions, test data, and expected results. Ambiguous instructions will create more review work, not less.
- Application access: The application must be reachable from Katalon TestCloud. Private environments may require Katalon Tunnel or IP allowlisting.
- Credential handling: Store credentials in TestOps secrets and variables instead of putting passwords directly in test steps or expected results.
- Failure review: Decide who investigates failed and unverified steps before connecting the results to release gates.
- Coverage boundaries: Use the runner for repeatable functional flows, but keep exploratory testing, risk analysis, accessibility review, and complex judgment tasks in the human testing plan.
Where it fits beside coded automation
AI Test Runner is not a replacement for every type of automation. It targets manual regression cases that matter to the business but have not been automated because scripting and maintenance costs were too high.
Teams with stable, high-volume checks may still prefer code-based tools such as Selenium, Cypress, and Playwright, particularly when they need detailed control over fixtures, network behavior, browser configuration, or custom assertions. A practical approach is to use coded automation for predictable technical checks and AI Test Runner for documented manual flows that remain expensive to run.
Availability and pricing
Katalon says AI Test Runner is available in Katalon TestOps, with AI usage included in the license cost. The company has not announced a separate public standalone price for the feature. Existing TestOps customers can start runs from TestOps test cases and suites, and Katalon also supports starting runs through its MCP server.
The launch moves AI testing beyond generating test ideas or converting requirements into drafts. Its value is more direct: preserve the manual knowledge a QA team already has, use agents to run repeatable checks, and let people review the evidence and make release decisions.
Looking for tools to match this read?
Jump into our independent side-by-side comparisons.
More in Software Tools
European Accessibility Act Enforcement in 2026: What QA Teams Need to Test For Now
The EAA has been enforceable since June 2025, with fines up to €1,000,000 per violation varying from country to country. Here's what accessibility testing teams should actually be checking as national enforcement ramps up.
The Flaky Test Crisis: Why Trunk, Datadog, and Meta Are Racing to Automate Flaky Test Detection
Flaky tests quietly erode CI trust and burn engineering hours. Here's how Trunk.io, Datadog, and Meta's internal tooling are using automated quarantine and probabilistic re-run analysis to fight back.
Synthetic Test Data Goes Mainstream: Why QA Teams Are Ditching Masked Production Copies for AI-Generated Datasets
Privacy law and breach risk are pushing QA teams away from masked production copies toward AI-generated synthetic test data. Here's why the shift is accelerating and which tools are leading it.