How to test your server infrastructure

How do you test that your server infrastructure scales correctly and can handle the real world? Let’s look at how to do it properly.

Written by Thaveshen Gopaul , Senior Software Engineer in Test at Gameye

When testing your infrastructure, you need a different mindset than when you test your game. It’s like the difference between testing an ice-cream flavour and testing an elevator. They just have very different criteria and approaches.

This can cause game developers a lot of problems when they make multiplayer games. So let’s look at what you need to know to test your infrastructure. And how to avoid the main pitfalls.

The three types of server infrastructure tests

In a game, your primary concern is whether it’s fun. But with infrastructure, you’re testing whether it’s safe and reliable. Anybody who tests infrastructure knows that everything breaks eventually. The key is finding out when it breaks. And what happens when it does?

Here are the three most popular infrastructure tests:

• Load testing. Put more and more weight into the elevator to find out how much it can take before it breaks. How much can it hold?
• Soak testing. Get the elevator to hold a typical weight and see how many days it can hold it for. How long can it last?
• Spike testing. Get people in the elevator to jump up and down and give it a good thrashing. What causes it to break?

All these tests apply to your network infrastructure, too. How many sessions can you run before your servers collapse? How long can you run your sessions before any memory leaks become a problem? What happens if you have a surge of players?

Using these tests you can figure out how stable your infrastructure is and come up with fixes ahead of time.

It all comes down to your planning

Make sure that you’re thoroughly planning out your tests. You want to come up with all the different possible scenarios. Testing infrastructure is all about considering the most extreme situations and planning accordingly. Trust us, those extreme situations will happen eventually. So ask yourself four questions:

1. What are our non-functional requirements

Non-functional requirements are the metrics that affect your players beyond basic system stats. The system might technically work, but is it a good experience? For example, it could be how long it takes to join a match, how quickly you can spin up a new session, or your in-game latency.

You can only get insights from the data you track. So make sure you consider what metrics would affect your player’s experience before you start testing.

2. What are your integration points?

Every third-party tool you use is going to affect your overall infrastructure in some way. For example, your matchmaker, your orchestrator and your player database. Ideally, you’d test everything all together. But that’s not always possible.

For example, it might be too expensive or complicated to run every component at once. In those cases, separate out your tests into more manageable chunks and try to use mock or stub components to make things easier.

Saying that, it’s best to try and budget for an environment that mimics your final release as much as possible, even if it’s a smaller scale. That way, you can be pretty sure things are going to work when you finally release.

3. How will you view your data?

Observibility is key. You want to make sure you’re monitoring and logging key metrics, but also that you have a way to understand the data. Set up dashboards to view the information in a more manageable way.

Having a clear way to analyze your data will make it much easier to spot bottlenecks and the root cause of any problems.

How do you get started?

There’s no right way to set up your testing environment. But if you want to make the process easier, we’d recommend using one of these open-source tools:

• Gatling. Gatling is a load-testing tool that supports multiple languages and plays well with both Docker and Kubernetes.
• K6. If you’re using Grafana to create dashboards, you’ll want to go with K6. It’s their own  load testing tool that hooks into the rest of their observability tools.

Which tool you choose depends on your architecture and what types of scenarios you want to run.

Get help with your infrastructure

If you’re looking to build your own server infrastructure, we can help you with the overall architecture, set up our orchestrator to manage it automatically, and help test it works. Just get in touch.

Meet the team at Gamescom

Our crew are heading to Gamescom this year, and would love to chat about the latest tech and games. Make sure to pop by and say hello. We're in hall 2.2, A040. Or drop a meeting in with our team.

See you there 👋