Originally posted on the Structure101 Blog.
Games are generally developed on an engine library that is developed by a separate team, on a different lifecycle. Upgrading to a newer version of the engine during game development can be a pressure point – there may be improvements that the game team want to make use of, but what will be the impact and effort for the upgrade, and when should this be tackled relative to the game release schedule?
Game developers are making use of a new Structure101 dependency database to manage engine usage and updates in a more informed and predictable way. I thought it would be interesting to outline the kinds of information they’re getting from the database so far, and how that information is helping them with engine upgrades…
- Where is the game bypassing the engine’s API? Restricting usage to the API increases the chances that upgrades will go smoothly.
- Where is the game using deprecated engine functions? These deprecated functions may be flagged as such in the currently-used version of the engine, or may have been deprecated (or even removed) in later versions. Catching usage of functions that are deprecated in the current or future versions of the engine, before new usages (i.e. game code changes) are committed, can avoid a lot of rework when the engine is later upgraded.
- How many engine usages will break the build if the engine is upgraded? Version control informs on where the engine code has changed since the version that the game is currently using, but which of those changes are breaking changes (like changed function signatures), and which of those are actually used by my game? A lot of compilation errors forces the game team to make a lot of game code changes at once, without the help of a buildable/testable target.
- Which areas or tests could be impacted by an upgrade? Understanding the functional impact on the game can help to focus testing, estimate upgrade effort, and decide the upgrade timing.
- Where have the game team created divergences in the engine code? Direct engine edits, or pre-porting of selected updates from future engine versions may be necessary to get the game to release. Monitoring the amount of divergence can help to avoid it becoming excessive, and finding out which divergences are in the call-tree of a failed test case can narrow the focus of debugging.
- Which engine APIs are not used by any games? Where the engine itself is also developed or extended in-house, unused functions or files can be removed from the API, and avoid unnecessary cost of upgrade/deprecation/test processes.
Having this kind of information at their fingertips has taken some of the work (and most of the guesswork) out of engine updates for our games development customers.
The Structure101 C/C++ parser
The key to providing complete and accurate answers to the above questions is accurate data about the dependencies that span the game and engine code. The dependency database is published from a high-fidelity C/C++ parser which:
- Is Clang based, but can handle code targeted for pretty much any compiler
- Is build-faithful, so that code is parsed with all the relevant compiler switches applied
- Detects dependencies through templates and macros
- Understands declarations vs. definitions, compile-time vs. link-time dependencies
- Identifies functions by their signatures (independent of line numbers), so that functions and dependencies can be related across builds
- Publishes dependency data to a Postgres database, deployable in a Docker image
- Provides an API/DSL for querying items, dependencies, dependency closures, comparisons between snapshots, etc.
- Is accessible via customisable web-application, or RESTfully
This parser has been developed in-house to catch the most detailed dependencies that span pre-compiler, compiler, and linker phases.
TL;DR
Queryable, high-fidelity dependency data for successive builds of games and engine is helping games teams do more informed, predictable and timely engine updates.
Chris Chedgey is a co-founder of Structure101, and has 30 years experience in commercial software development, notably on large military and aerospace projects in Canada, including 5 years on the International Space Station (ISS).
He is a strong advocate for the importance of good software structure, and has spoken at many user groups and conferences including JFall, WICSA, Oredev, JavaOne, Houston Techfest, JAX, Javaland, 33rd Degree, JFocus, JAX Finance and Devoxx.
