Como você define o escopo e os objetivos dos testes de sistema?

System test is not just limited to software. We use this term for hardware and firmware testing as well. Not only are we testing for good path functionality but also serviceability and recoverability of the system with bad path testing using error injection tools or other means to inject faults. This allows us to test the stability of the system such the end user encounter any problems.

System testing is a vital technique for ensuring that a product meets its required specifications. It helps to identify potential issues, such as: Functionality: Whether the software is performing correctly, or if any functionality is missing or incorrect Compatibility: How well the software works in different environments Recoverability: How the software responds when it fails Data access: Whether the software can access data securely, and if it's exposed to unwanted resources User experience: How well the software is used, and if it meets user expectations

System testing is critical because it ensures that the entire system works as intended across various components and environments. I’ve seen firsthand how thorough system testing can catch issues that may have slipped through earlier stages of testing, like integration or unit testing. It’s the final validation before a product goes live, making sure that everything—user interface, databases, APIs—functions cohesively under real-world conditions. A well-executed system test gives confidence that the product is ready for users.

"As a tester, I believe system testing plays a pivotal role in guaranteeing the robustness and functionality of a software system. It serves as the final checkpoint before deployment, validating that all components work harmoniously and meet the specified requirements. Through systematic testing, we can identify and rectify defects, ensuring a smoother user experience and instilling confidence in the system's performance. It's not just about finding bugs; it's about delivering a reliable and high-quality product to end-users.

In my experience, the terminology System Testing is widely used for Functional Testing which is an intermediate, but crucial, phase in the test lifecycle of a project just following the Unit Testing. System Test is the phase where individual components of a system are tested in conjuction.

For IBM Z the scope of our effort for each system test cycle is that 80% of the effort goes to validating there is not breakage in any existing function and 20% of the test effort goes towards validating new function.

I start by identifying all the system’s components that need to be tested, including hardware, software, and integration points. For example, in a web application project, I focused on testing the front-end, back-end services, databases, and third-party integrations. The scope must cover both functional and non-functional aspects, such as performance, security, and usability. Clearly defining the scope helps ensure that the testing is thorough and aligns with the project’s requirements.

Here's an example of system test scope for an e-commerce website: Functional areas: User registration and login, product search and filter, shopping cart, checkout process, payment gateway integration, order history. Non-functional aspects: Load testing for peak traffic, security testing for payment gateway, compatibility testing on major browsers and devices. Interfaces and integrations: Integration with third-party payment gateways (e.g., PayPal, Stripe), and shipping service APIs. Out-of-Scope: Marketing emails and notifications, third-party analytics integration.

An example of a tool ( or means) to determine the scope of any test phase is a Test Risk Assessment questionnaire. Any organization based on their risk appetite must enable the Change Delivery Framework by defining the process and providing the tool as a means of determining the scope of testing for each test phase.

The objectives of system testing should align with both the project goals and the user’s needs. In my experience, objectives usually include validating the system’s performance under normal and peak conditions, ensuring security measures are in place, and confirming that all features work as intended. For instance, if the system is an e-commerce platform, key objectives would include testing the checkout process, payment gateway integrations, and load handling during peak traffic. Defining these objectives helps prioritize testing efforts and ensure critical areas are thoroughly tested.

We can begin by thoroughly understanding the system requirements. System testing is intended to ensure that the software meets these requirements in terms of functionality, performance, security, usability, and other specified criteria. we can identify the critical business processes or workflows that the system is expected to support. Understanding these core processes helps in defining test scenarios that mimic real-world usage.

Here's an example of objectives for system testing of an e-commerce website: Verify functionality: Ensure all e-commerce functionalities work as intended. Validate performance: Confirm the application can handle high traffic during peak shopping times. Check security: Identify and resolve security vulnerabilities in the payment process. Ensure usability: Validate that the user interface is intuitive and user-friendly. Assess compatibility: Ensure the application works seamlessly on different browsers and devices. Validate integration: Ensure smooth integration with payment gateways and shipping services.

The whole area of Test Objectives seems more complicated than it needs to be. The Test Objectives are not a thing unto themselves but rather a derivation from whatever requirements are being verified. At an atomic level, the objectives should be to verify the associated requirements and not much else. KISS! Your stakeholders will thank you!

Once testing is completed, I create a system testing report to document the process and results. I include key information like test cases, testing methodologies, issues found, and their resolutions. Tools like JIRA, TestRail, or even Word can be used to compile the data, and then I export the document into a PDF format for easy sharing. This report is essential for providing stakeholders with a clear summary of what was tested and any remaining issues that need to be addressed before release.

To create a system testing report in PDF format, start by compiling all relevant test results, including test cases executed, pass/fail status, defects found, and overall system performance. Use a tool like Microsoft Word or Google Docs to format the report with sections such as Introduction, Test Scope, Test Approach, Test Results, Defects Summary, and Conclusion. Once the content is ready, convert the document to PDF format. Most word processing software allows you to save or export the file as a PDF directly. Alternatively, you can use online converters like Smallpdf or Adobe Acrobat to convert the document to PDF. This ensures a professional and easily shareable format for the system testing report.

A system testing report serves as a critical communication tool between the testing team and other stakeholders, including project managers and developers. I’ve used it to highlight issues that need immediate attention and to justify additional testing resources if necessary. It also acts as a reference document for future development cycles, helping teams understand what worked and what didn’t. Additionally, the report can serve as proof of compliance for regulatory or client requirements.