AI BS In Testing, Playwright MCP, k6 Studio,and More

Did you hear about the must view resource on Modern-Day Oracles or Bullshit Machines: How to Survive in a ChatGPT World?

Have you seen the new the Playwright Model Context Protocol (MCP)?

What is k6 studio?

Find out in this edition of the Test Guild New Shows Newsletter for the week of March 23.

So, grab your favorite cup of coffee or tea, and let's do this.

News Show Sponsor ❤️

Shout out to our sponsor ZAPTESTAI—a platform you'll want to check out if you're serious about automation. Their AI-powered tools, like Plan Studio, turn your ALM data into reusable, automation-ready modules without the usual headaches. At the same time, Copilot eliminates tedious tasks like script generation and object management. The best part? They're offering a 6-month No-Risk Proof of Concept, so you can test-drive the platform and see actual results before committing.

If you want to level up your automation, 👉 ZAPTEST.AI is worth a serious look!

Cypress Introduces cypress.stop() to Improve Test Run Control

Cypress has released a new feature, 👉 cypress.stop(), designed to give testers more precise control over test execution within a spec file. This command halts further test execution at any point in a test run, allowing developers and QA engineers to stop tests programmatically based on custom logic or unexpected conditions.

This update follows the earlier introduction of Auto Cancellation, a Cypress Cloud feature that automatically halts all parallel test runs when a new commit is pushed, ensuring CI resources aren’t wasted running outdated test builds.

The key distinction is scope:

• cypress.stop() operates within a single spec file during local or CI runs.
• Auto Cancellation applies across all parallelized test jobs in the cloud environment.

The cypress.stop() command is particularly useful during debugging or in conditional test flows where continuing execution could result in misleading or irrelevant test outcomes. It does not throw an error or fail the run — it simply ends it.

Grafana Launches k6 Studio to Streamline Performance Testing

Grafana Labs has announced the general availability of 👉 k6 Studio, a new open-source application aimed at simplifying the process of creating performance tests. Designed for software testers, developers, SREs, and QA professionals, k6 Studio enables users to record API interactions and automatically convert them into structured test scripts compatible with the popular k6 performance testing tool.

The application also includes a rules-based system for modifying test scripts—supporting features like data extraction and parameterization—and allows users to run their tests directly in Grafana Cloud k6. The goal: reduce the technical friction often associated with writing performance scripts from scratch, encouraging broader adoption of continuous performance testing across teams.

The launch follows a growing demand for tools that accelerate test creation and reduce the overhead associated with performance testing, especially in fast-moving DevOps and CI/CD environments.

Software Testing has a New AI Literacy Resource for Testers

Michael Bolton is urging testers to explore a newly released resource that dissects the impact of generative AI on critical thinking and decision-making. The work called 👉👉 Modern-Day Oracles or Bullshit Machines: How to Survive in a ChatGPT World, was created by University of Washington professors Carl Bergstrom and Jevin West, known for their widely respected book and course Calling Bullshit.

The new release, presented as a website rather than a traditional book, provides an accessible overview of AI systems—particularly large language models like ChatGPT—and their influence on education, public discourse, and intellectual rigor. Structured with short chapters, discussion prompts, and embedded videos, the content is designed to spark reflection and enhance digital literacy.

Michael connects the material directly to the core responsibility of software testers: applying critical thinking to software systems and the environments in which we operate.

He argues that while all team members may exercise judgment, testers are uniquely tasked with scrutinizing and exposing issues—making resources that strengthen analytical skills particularly relevant.

Playwright Model Context Protocol (MCP

Microsoft has officially introduced the 👉 Playwright Model Context Protocol (MCP), a technical advancement aimed at bridging browser automation with AI-powered testing workflows. The protocol allows Large Language Models (LLMs) to interact with web applications using structured accessibility snapshots, rather than relying on visual pixel inputs or computer vision techniques.

MCP builds on Microsoft’s existing Playwright framework and introduces a standardized interface for LLMs to retrieve semantic context from the Document Object Model (DOM). This means that instead of interpreting raw HTML or rendering a page visually, AI models can now access a well-organized structure of the page's elements, including roles, labels, and states—mirroring how assistive technologies access content.

This structured context enables LLMs to perform more reliable and explainable interactions with web elements, potentially improving the quality and maintainability of automated tests driven by AI. Microsoft notes that MCP could accelerate the development of natural language-based test generation, automated bug reproduction, and accessibility validations.

What is MCP?? 

Still hearing all the buzz around MCP and not sure what it even means? If that’s you here is an excellent resource to get up to speed

This is 👉 a Deep Dive Into MCP and the Future of AI Tooling by Yoko Li

She explains the The Model Context Protocol (MCP), introduced in November 2024, which is an open standard designed to streamline interactions between AI models and external tools, data sources, and services.

By providing a unified interface, MCP enables AI agents to autonomously select and integrate various tools to accomplish tasks, reducing the need for custom integrations.

Developers have begun implementing MCP across various applications. For instance, the code editor Cursor utilizes MCP to transform into a multifaceted platform capable of sending emails via the Resend MCP server, generating images through the Replicate MCP server, and integrating with services like Slack. This flexibility allows developers to manage tasks directly within their integrated development environments (IDEs), enhancing efficiency.

Despite its advantages, MCP faces challenges, particularly in areas like authentication, authorization, and server discoverability. Addressing these issues is crucial for MCP's broader adoption and its potential to become a standard in AI-tool interactions.

Vibe Development

Another term I’ve been hearing more and more about and seeing memes is around Vibe Development. What is it

 The term 👉 "vibe coding" has emerged, referring to the practice of using artificial intelligence (AI) tools to generate code based on high-level descriptions provided in natural language. This approach allows individuals to create applications by interacting with AI models, which then produce the corresponding code. Advocates suggest that this method lowers the barrier to software development, enabling those with limited programming experience to build functional software.

However, concerns have been raised about the reliability and maintainability of AI-generated code. Critics argue that while AI can produce code snippets quickly, the resulting code may lack optimization and contain errors that are difficult to detect without a deep understanding of programming principles. There is also apprehension regarding the security of such code, as AI models might not adhere to best practices, potentially introducing vulnerabilities.

While AI-assisted coding, or "vibe coding," offers a novel approach to software development, software testers should be vigilant about the potential for suboptimal and insecure code

Agentic AI Hype 

In a recent LinkedIn article, by Tariq King, AI expert and Head at Test IO, critically 👉 examines the current enthusiasm surrounding "Agentic AI." He contends that the concept of AI agents is not a novel development but has been foundational to artificial intelligence since at least 1995, as detailed in the seminal textbook "Artificial Intelligence: A Modern Approach" by Stuart Russell and Peter Norvig.

Tariq highlights that intelligent agent—systems capable of perceiving their environment, making decisions, acting upon those decisions, and learning from experiences—have long been integral to AI advancements in areas such as robotics, multi-agent systems, and virtual assistants. He expresses concern over the software testing community's role in amplifying the hype around Agentic AI, noting instances where AI-driven testing tools have been promoted despite limited efficacy.

While acknowledging the enhanced capabilities brought by large language models like GPT-4, Claude, and Gemini, Tariq emphasizes that the vision of fully autonomous, general-purpose AI agents remains speculative. He advocates for a balanced approach, recommending the integration of AI co-pilots and agents within well-defined tasks, supervised by human oversight, to effectively augment intelligence rather than pursuing unattainable autonomy.

Tariq has been talking about AI in testing before it was a BIG thing as you can see by a session, he did for the TestGuild way back in 2019 :)

Software testers should critically assess the current discourse on Agentic AI, recognizing that while AI agents have evolved, the core concept is longstanding, and claims of groundbreaking autonomy may be overstated.

That's a Wrap

So that's it for this Test Guild News Show Newsletter edition.

Make sure to 👉 subscribe to never miss another episode.

I'm Joe Colantonio, and my mission is to help you succeed in creating end-to-end full-stack DevSecOps automation awesomeness.

As always, test everything and keep the good.

Cheers!

Next Steps

Join our 👉👉👉 private Automation Testing Community and get access to other like minded experts 24x7.