Mutation Testing, Playwright CLI, AI Test Agent and More

What is mutating testing?

Have you seen the new CLI Tool for Playwright Tests

How you make your own AI Test Agent

Find out in this edition of the Test Guild New Shows Newsletter Recap for the week of Feb 16th.

ZAPTEST.AI

First I wanteded to welcome our new sponsor ZAPTESTAI– a platform I think you’ll want to check out if you’re serious about automation. They’ve got some pretty slick AI-powered tools, like Plan Studio, which helps you turn your ALM data into reusable, automation-ready modules without the usual headaches. Plus, their Copilot feature knocks out repetitive tasks like script generation and object management, so your team can stay focused on high-impact work. What I really like? They’re offering a crazy 6-Month No-Risk Proof of Concept, so you can test-drive the platform and see real results before committing. If you’re looking to level up your automation game, 👉 ZAPTEST.AI is worth a serious look.

Automation Guild 2025

It’s been a crazy few weeks around here with me focusing on our Annual Automation Guild conference. Last week was a blast and happy to report our 9th annual event was a HUGE success.

Also wanted to let you know that we got so much demand we decided to keep registration open so folks can 👉👉👉 now get instant access to all the awesomeness that took place last week.

You’ll get access to all the recordings of the event that took place last week as well as access to our yearlong community where you’ll receive one new training a month on a topic the Guild chooses.

Playwright + Bolt Framework

In this article 👉 Playwright Test Automation Framework Developed with The Help Of AI Search Agent. Jay Kishore Duvvuri explains how to generate a frontend test automation framework with Playwright and JavaScript on saucedemo website using AI Search Engine called Bolt.

This is the first time I've heard of Bot bolt.new developed by the StackBlitz team, is a cool and powerful AI tool designed to streamline web development by automating code generation. It is a powerful AI tool designed to streamline web development by automating code generation.

Jay shows how this AI Powered tool helps you generate, test, and deploy web apps that use modern frameworks.

Mutating testing

In a recent blog post, software professional Mahak Tripathi Tripathi delves into 👉 the concept of mutation testing, highlighting its significance in maintaining high code quality. Mutation testing is a method used to evaluate the effectiveness of software tests by introducing small changes, or "mutants," into the code to ensure tests can detect faults. Mahak emphasizes that this approach helps reveal inadequacies in existing test suites, making it a crucial component in the development process.

They outline the benefits of mutation testing, primarily its ability to enhance test coverage and identify gaps that traditional testing methods might overlook. This technique challenges the robustness of test cases, pushing developers toward creating more resilient and comprehensive test strategies.

The post underscores the importance of integrating mutation testing into the software development lifecycle, suggesting that its adoption can lead to the development of more reliable and error-free software products. Despite the computational resources it requires, this testing method is presented as a valuable tool for software testers aiming to elevate the precision of their testing outcomes.

I think software testers should consider incorporating mutation testing into their quality assurance processes to identify weaknesses in test suites and enhance overall code quality.

New CLI Tool Enhances Playwright Test Execution

Dennis Bergevin, a Software Quality Engineer and Cypress Ambassador, has developed 👉playwright-cli-select, a command-line utility aimed at improving test execution for Playwright users.

This tool introduces interactive CLI prompts, enabling testers to select and run specific specs, tests, or tags directly from the terminal, thereby simplifying the management of extensive test suites.

The utility also offers a customized playwright test command, allowing users to pass desired arguments for tailored test runs. Notably, it supports features like running the last failed tests and selecting only changed tests, enhancing testing efficiency. The tool is available on GitHub under the MIT license, providing an open-source solution for the Playwright community.

Quality Nonsense: Analyzing a Test Suite Using The Home Edit

In her latest blog post Judy Mosley dissects 👉 common missteps when evaluating test processes and emphasizes the importance of a robust quality assurance framework.

Key insights include identifying inefficiencies in traditional testing methods and stressing the necessity for adaptive, context-aware practices that align with project-specific demands.

Among the critical discussions is the misapplication of standardized metrics which often leads to a false sense of security about software quality.

The analysis underscores the significance of integrating exploratory testing into the process to uncover hidden defects that structured tests might overlook.

The article further suggests that an over-reliance on automated tests without thorough validation can result in overlooking critical bugs, ultimately undermining test reliability.

FRIDAY - AI Test Agent

Dipjyoti Metia a Chapter Lead – Testing has unveiled 👉 Friday, an evolving open-source project aimed at automating various aspects of software testing. Friday leverages large language models (LLMs) such as OpenAI, Gemini, or Mistral to generate test cases, thereby reducing the manual effort traditionally required in test creation.

Key functionalities of Friday include:

• Automated Test Case Generation: Utilizing LLMs to produce relevant test scenarios.
• API Testing Agent: Employing LangChain’s agents to test APIs directly from OpenAPI specifications.
• Requirement Extraction: Deriving requirements from Jira tickets or GitHub issues to ensure alignment between development and testing.
• Contextual Integration: Fetching additional context from Confluence pages to enrich test cases.
• Document Management: Storing and searching documents using ChromaDB vectorization for efficient information retrieval.
• Flexible Export Options: Allowing test cases to be exported in JSON or Markdown formats for seamless integration into existing workflows.
• Web Crawling for Context: Creating embeddings with a web crawler to gather supplementary context for testing.

The project is actively seeking feedback and contributions from the community to refine and expand its capabilities. Interested individuals can access the repository on GitHub.

"Promptwright"

Promptwright is 👉 an AI tool designed for browser automation. It transforms simple English instructions into automated workflows for efficient testing.

It features a user-friendly interface incorporating top AI providers and allowing users to access powerful tools easily. The tool supports flexible browser execution, enabling local or remote testing and compatibility with frameworks like Playwright, TypeScript, Python, and Cypress.

Promptwright automates multi-page workflows, demonstrated through the automation of login and contact creation using AI-powered scripts. It gathers page information and executes workflows accurately, generating robust automation code and filling forms with relevant data. The tool produces real-time TypeScript code for documentation and collaborative sharing, making it easy for teams to integrate and utilize the generated scripts.

Additionally, users can download or copy the generated code and customize a comprehensive table of interactive elements, including CSS and XPath locators, which can be exported as a CSV file.

Overall, Promptwright simplifies browser automation for testing, making it accessible with just a click to start a new task. It is also available on GitHub to enhance automation workflows.

mabl Pioneers New AI and Playwright Integrations to Streamline Workflows

In a recent blog post, a new software testing framework, TestAdvantage, was introduced, promising to enhance automation testing workflows. 👉 TestAdvantage integrates with popular development environments, offering seamless compatibility with DevOps practices.

Testing teams are likely to benefit from its accelerated test execution times and increased support for cross-platform testing, addressing a prevalent challenge in the software industry. Moreover, TestAdvantage introduces AI-driven analytics to optimize test coverage, providing actionable insights for QA managers.

That's a Wrap

So that's it for this Test Guild News Show Newsletter edition.

Make sure to subscribe to never miss another episode.

I'm Joe Colantonio, and my mission is to help you succeed in creating end-to-end full-stack DevSecOps automation awesomeness.

As always, test everything and keep the good.

Cheers!

Next Steps

Join our private Automation Testing Community and get access to other like minded experts 24x7.