How to Recognize Cargo Cult DevOps

When I read that an organization “uses DevOps”, I am skeptical.

Often it means that they are using Jenkins. Or that they are developing “in the cloud”. And that’s all.

They might have component pipelines. (See my article on why “DevOps is not a pipeline”.) And they might manage their unit test coverage. And they might have “autonomous self-organizing teams”.

But that is not DevOps. It is “cargo cult” DevOps - that is, going through some of the motions, but not really doing it, or doing it right or with the actual intent of DevOps.

In fact, if you look at what Amazon, Google, Netflix, and other companies like those do - the companies that invented DevOps methods - they do things very differently from what is advocated by run-of-the-mill articles about DevOps.

For example, at Google, a code commit does not succeed until integration tests have passed. Integration tests. There is no merge into a shared branch until the code integrates. Do your teams do that?

And while most of these organizations have semi-autonomous mostly-self-organizing teams, the operative words are “semi” and “mostly”. For example, Netflix has a huge tools team that assists development teams in getting automation set up. And the tools team has created a huge suite of self-service tools.

Sure, if you have that level of support, then you can approach being autonomous. And self-organizing? These organizations also have team lead roles.

I started to reflect on what makes an organization DevOps. This is challenging since there is no definitive definition, and there should not be, because all DevOps practices are contextual: you don’t just apply them blindly. Nevertheless, I came up with a list that, if one scores high against this list, then one is probably doing DevOps. Here it is:

• You have integration pipelines.
• You have dynamically provisioned int test envs.
• You do not have target dates for each level of integration.
• You continually measure and optimize your end-to-end customer-facing product delivery flow.
• An end-to-end integration test can be triggered on demand, in an isolated manner, to test code changes, without affecting any other developers or testers.
• Developers can easily run integration tests locally - they are not limited to running them in Jenkins as part of a pipeline.
• You fine tune and customize security IDS rules for your apps.
• You easily do customer-facing A/B experiments and collect data on those.
• You have an end-to-end dashboard that shows both the business and operational performance of your customer-facing products.
• You merge after you integration test.
• You don’t focus on unit tests, but instead focus on a range of product tests.
• You perform automated failure mode testing, and have automated test cases for your resiliency mechanisms.
• You frequently test in the prod network, or a production-like network configuration that has the same region/AZ/firewall topology as your production network.

If you are doing most of these things, I think it is safe to say that you are “doing DevOps”. If you are doing few, then you have a ways to go.