Interview #123: TestNG - How do you rerun only failed tests in a suite?

In real-world test automation, especially in large test suites, it's common for some tests to fail intermittently due to reasons like network latency, environment instability, or third-party API issues. Re-running only failed tests instead of the entire suite saves time and resources. TestNG provides a built-in mechanism to support this rerun strategy using its test result tracking and rerun capability.

Disclaimer: For QA-Testing Jobs, WhatsApp us @ 91-9606623245

Below is a comprehensive explanation of how to rerun only failed tests in a suite using TestNG:

✅ Step-by-Step Process to Rerun Failed Tests in TestNG

1. Run the Test Suite Normally

When you first execute your TestNG suite (using either testng.xml or a test runner), TestNG will automatically generate several output files and reports, one of which is a file called testng-failed.xml.

mvn test        

2. Locate the testng-failed.xml File

After execution, TestNG generates a file at this path:

<project-folder>/test-output/testng-failed.xml        

This XML file contains only the failed test methods from the previous run. It is dynamically created by TestNG and is structured like a regular testng.xml file, which makes it executable directly.

3. Rerun the Failed Tests

You can now execute the failed tests by pointing TestNG to the testng-failed.xml file.

Option 1: Using TestNG in the IDE

• Right-click on testng-failed.xml → Run As → TestNG Suite

Option 2: Using Maven

If you are using Maven, you can run it like this:

mvn test -DsuiteXmlFile=test-output/testng-failed.xml        

This will rerun only the failed tests, as defined in that file.

🔁 Rerun Failed Tests Automatically – IRetryAnalyzer

Sometimes, you may want failed tests to be retried automatically a few times before being marked as failed. TestNG supports this via the IRetryAnalyzer interface.

Step 1: Create a Retry Analyzer Class

import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;

public class RetryAnalyzer implements IRetryAnalyzer {
    private int retryCount = 0;
    private static final int maxRetryCount = 2;

    @Override
    public boolean retry(ITestResult result) {
        if (retryCount < maxRetryCount) {
            retryCount++;
            return true;  // retry this test
        }
        return false;  // do not retry
    }
}        

Step 2: Attach RetryAnalyzer to Your Tests

You can either:

• Add it directly to each test method:

@Test(retryAnalyzer = RetryAnalyzer.class)
public void testAPIEndpoint() {
    // test logic
}        

• OR use a TestNG Listener to apply it globally (advanced approach).

📁 Important Notes

• testng-failed.xml gets overwritten each time you run a new test suite. To preserve it for reruns, you might want to copy it manually or configure your CI to save artifacts.
• If a test fails again in the rerun, it will appear in the next testng-failed.xml too.
• This feature works only if the test-output folder is preserved (do not delete after test runs).
• This rerun strategy is very useful in CI/CD pipelines to stabilize flaky tests.

✅ Best Practices

• Use IRetryAnalyzer to automatically retry flaky tests before marking them failed.
• Set a limit on retries (e.g., 2 or 3) to avoid infinite loops or excessive resource usage.
• Combine rerun logic with test reporting tools like Allure, ReportNG, or ExtentReports for better visibility.
• Clean up environment-specific issues to reduce reliance on reruns.
• In Jenkins or CI, store the testng-failed.xml as an artifact and rerun via post-build action or separate job.

Conclusion

TestNG offers a powerful and convenient way to rerun only failed tests using the automatically generated testng-failed.xml file. This capability, along with the IRetryAnalyzer interface, allows test engineers to create robust and efficient test suites by minimizing rerun time and isolating flaky failures. Whether in local development or continuous integration environments, this feature helps maintain test suite stability and improves feedback loops.