Testing in a Continuous Deployment World

On February 08, 2017, I went to the Continuous Delivery Meetup in Amsterdam. There was a talk on ‘Testing in a Continuous Deployment World’ by Wouter Lagerweij

The talk was mainly about making the move from Continuous Delivery to Continuous Deployment, the changes it demands and the (non-business related) benefits it brings. 

In a nutshell

Continuous Deployment forces and enables everyone to follow the agile principles to the fullest. There is no skimping on things like Definition of Ready and Definition of Done. The talk also introduced the concepts of Hypothesis Driven Development and Infrastructure Testing. 

Now here is the tl;dr version

This is what was, at least for me, new in the talk

Business Tests

This is a name given to the Lean concept of Minimum Viable Product and it’s proper use in the software world.

MVP is the vehicle for business to test a hypothesis; is what we think will bring value to them, their customers and their users (or a combination of these)?

Hence these are also called Hypothesis Driven Development. We need to not see the MVP as a milestone; a first release. CD is critical in getting quick feedback to business from the MVP.

Infrastructure testing. 

He threw terms like unit testing for deployment scripts (puppet, chef etc.) and validations of docker containers. He did not elaborate on these.

This is what was, for me, not-so-new in the talk

• CD demands that acceptance tests are automated and written in the same commit that brings in the functionality. The upside of this is that it brings the testing mindset to everyone in the team and brings the tester earlier in the process.
• CD makes the developers think more about the code they write and the behaviour that is expected of the commit they are going to make.
• CD demands that all sunny and rainy day scenarios that can be expected. Without this automated acceptance tests cannot be written, or worse, we do not have sufficient test coverage.
• CD promotes the practice of ‘three amigos’. This is a practice where a QA, a developer and the Product Owner sit down to discuss every story and reach a common understanding on the same.
• CD makes the tester become a much more integral part of the team. It promotes the spread of the art and craft of testing and the way of thinking about it in the rest of the team.
• CD brings the monitoring and operations aspects closer in the team.
• CD takes you to the stage of exploratory testing on production because there is no test environment left to test on.