Quality in the age of AI

Orcastrator is an agentic QA automation system built to help teams keep software quality aligned with the speed of AI-assisted development.

For the demo, I’ll start without Orcastrator.

I’ll first show how a builder would typically approach AI-assisted testing today: using Playwright, integrating testing workflows into the IDE, and using agentic coding tools to generate and refine test cases against a real application. This will include the practical flow of identifying what needs to be tested, writing Playwright tests using Playwright Agents / MCP, running them, debugging failures, and trying to understand what those tests actually say about the health of the product.

From there, I’ll show the limitation we ran into: writing tests faster is useful, but it does not automatically give you a structured understanding of your application’s quality.

The more important layer is understanding the product as a set of meaningful application areas, mapping those areas to the test cases that verify them, and then using that mapping to report on the health of the product as a whole.

That is the engineering problem Orcastrator is designed around.

In the final part of the demo, I’ll show how Orcastrator and its MCP workflow make this process faster and more systematic: moving from isolated test generation to a quality model where application areas, test cases, execution runs, logs, screenshots, traces, and product health reporting are connected together.

The goal is not to present a product walkthrough, but to show the testing workflow before and after introducing a dedicated agentic QA layer.