Operational Excellence - #1

Building high-quality software requires rigor and a firm commitment to excellence. My deep belief in the importance of quality assurance (QA) drives my teams to implement robust practices to ensure their products meet the highest standards before reaching customers. To maintain the highest bar of quality, my teams must employ diverse mechanisms: a comprehensive and robust workflow, a multitude of tests (unit and end-to-end), meticulous reviews (code, security, and performance), and bug COE (Correction Of Error).

I have been working and iterating with my teams on this topic during the last years and come up with an approach that works for us. Let's look closer at the workflow we adhere to. It is divided into distinct steps which are covering:  

• Setup: the feature is defined and specified, potential impacts on performance, security or existing components are identified and architecture choices are described. The key in this step is to build the right feature, the right way. Scalability and stability of products that do not deliver the expected outcomes are useless.
• Validations (Setup, Feature, Code and Security): we ensure that the development is consistent with the setup, contains no bugs, performance degradations, or introduced security risks mixing manual and automated mechanisms. The task owner will seek advice and guidance from other team members, fostering knowledge sharing and team building. They have the ownership of the work done and its impacts. During this validation process, modifications may be requested, and thoroughness is highly encouraged since this phase is crucial to ensure no regression or bug is introduced into the product. During code validation, security guardians are involved to ensure that the code complies with our security standards.
• Environment testing: all environments (preproduction, productions, betas, gammas…) validation are continuously tested manually. Preproduction is additionally automatically scanned and tested daily. We strive to be one step ahead of the client in identifying bugs by catching them before reaching the production environments. We count over thousands of tests running with Cypress, Playwright, Junit and other custom business tools for application with daily team reports.
• Documentation: thorough documentation accompanies all new features. It serves a dual purpose: empowering our customers with the information they need for efficient and transparent usage, and communicating the benefits of new features and their adoption.

At first sight, this workflow may appear complicated and time-consuming, but this investment pays off daily. Thanks to this commitment to quality, we are continuously improving our software while encountering less than one issue per week among thousands of active users. This workflow also allows us to anticipate potential incoming problems or challenges to overcome.

Fixing issues is a major point which is considered in our workflow. Issues arise from various sources (e.g. users and ticketing systems). To ensure issues are fixed and prevent recurrence, we meticulously describe each issue, identify the scenarios and impacts, and schedule meetings with the involved engineers to discuss the COE. The main points are: What happened? Why? And how to avoid this happening again? Our aim is to identify the root cause, create generic solutions, and reduce the number of similar issues permanently. Identifying issues is a key point in our quest for quality, especially when our aim is to identify and address them before our customers do. To achieve this, we orchestrate monthly BugBash sessions, where the entire team collaborates to “break” the application. We've found this team-building exercise to foster team cohesion while purposefully challenging our product's integrity. All major findings are prioritized and addressed in the following days, if not hours. 

In my next articles, I’ll share more details on each of the steps from setup to preproduction. I’ll also share details on the remaining part (from preproduction to production).  

Thanks for reading!