Lessons from The CrowdStrike-Microsoft Incident… and How to Safeguard Your IT Systems against Failed Updates

Well, I was going to talk to you about layered cybersecurity techniques today, but if I did that you'd be saying 'so what Ed? We want your two cents on CrowdStrike!' So here it is.

The recent CrowdStrike-Microsoft incident reminds us that threats to our IT systems don't always come from hooded cyberpunk criminals. Sometimes, they stem from seemingly innocent acts gone awry. This event perfectly illustrates why layered IT security is so critical - because danger can lurk in unexpected places.

So What Happened?

Basically, a software update went terribly wrong. On July 18, leading cybersecurity firm CrowdStrike pushed out an update to its Falcon sensor (a lightweight software agent installed on endpoints like computers and servers to detect and prevent security threats in real-time). This update contained a C++ coding error that created a null pointer, i.e. it essentially pointed to nothing in computer memory. So, when the code tried to access this non-existent data, it triggered a system crash, resulting in the infamous Blue Screen of Death on Windows systems.

This seemingly simple error had a domino effect. As the faulty update rolled out, it affected millions of devices running Microsoft Windows, bringing critical services like Microsoft 365 and Azure to a standstill… and you know the rest! 

It's a stark reminder of how interconnected our digital world is.

The Impact: You Know All This, But How Did It Impact Businesses?

The outage wasn't just a minor inconvenience - it caused widespread disruption across industries. From healthcare services to airlines, stock exchanges to small businesses, the impact was felt globally. SMEs and global multinationals found themselves struggling to maintain operations as their IT systems went dark.

But you already know all of this, so what can we glean from it? And how can YOU protect YOUR IT systems?

As I said, the incident exposed the vulnerability of our interconnected digital ecosystem, even from a seemingly innocent C++ coding error, and that not all IT threats stem from cyber criminals out to ransom you for millions. 

Nothing underlines the non-negotiable need for robust safeguards more than a single update that brings down global systems. And, let’s not forget, having that well-rehearsed contingency plan to hand for when things do go awry.

What We Can Learn from Coding The Error that Brought Down The Internet

Alright, let's turn this digital lemon into cybersecurity lemonade. The CrowdStrike snafu wasn't just a headache—it was a wake-up call with a bullhorn. So here are the takeaways that might just save your bacon one day:

1. Have Robust Vendor Management

Do your due diligence on your tech partners and keep clear communication channels i.e. know who to call when things go sideways.

2. Implement Comprehensive Testing and Deployment Strategies

Staging environments exist for a reason, so use them to test updates before they hit your production systems. And please, implement phased rollouts! It could save you from a company-wide meltdown.

3. Practice Your Incident Response Plan

Regularly reviewing and updating your incident plan isn’t enough - do your drills and training too! When a crisis hits (yes you read correctly: when), you'll be glad you practiced it.

4. Seek out Diversification and Redundancy

Minimize single points of failure in your IT systems, or if they can’t be avoided, keep a close watch on them and have a back up plan if they do encounter issues. For example, consider multi-cloud or hybrid infrastructures to spread your risk.

5. Continually Monitor and Log Events and Incidents

Use real-time monitoring tools to catch issues as they happen. Logging is also your forensic friend, i.e. when problems occur, you'll want to know exactly what happened and why.

Closing Thoughts: An Ounce of Prevention For A Pound of Cure

Look, I get it. You're juggling a million priorities and cybersecurity might feel like just another item on your never-ending to-do list. But incidents like the CrowdStrike-Microsoft fiasco remind us why we can't afford to be complacent.

Proactive cybersecurity measures aren't just a nice-to-have - they're essential for the survival and growth of your business. Stay curious, keep learning, and remain vigilant. The digital landscape is always shifting, and we need to shift with it.

If you’re reading this then you know I specialize in cybersecurity solutions for companies like yours. So, if you want to chat about how I can help bulletproof your systems then let’s schedule a meeting and talk shop.

Remember, in the world of IT security, an ounce of prevention is worth a pound of cure. More on layered cybersecurity approaches and tactics next time. Stay safe out there, folks!

Ed