How to Fix Hundreds of Bugs in Legacy Code and Not Die (Unreal Engine 4)

How to Fix Hundreds of Bugs in
Legacy Code and Not Die
(Unreal Engine 4)
George Gribkov

About Me:
George Gribkov
A C++ developer and developer of the PVS-
Studio static code analyzer
I help our clients fight errors
I write articles on looking
for bugs in open-source
projects.
gribkov@viva64.com
2

 How to fight bugs in legacy code?
Overview
3

 Should you do it at all?
Overview
4

 What to do if there are hundreds or even thousands
of errors?
Overview
5

 What to do if there are hundreds or even thousands
of errors?
(that’s usually the case)
Overview
6

 What to do if there are hundreds or even thousands of errors?
 How to avoid spending a plethora of man-hours on this?
Overview
7

 What to do if there are hundreds or even thousands of errors?
 How to avoid spending a plethora of man-hours on this?
 And still, how did you work with Unreal Engine?
Overview
8

Should You Fight Bugs in Legacy Code?
10

 Old bugs are still bugs
 Old vulnerabilities are still vulnerabilities
 A charged gun will shoot you in the foot
eventually
Why?
12

14
14
Ariane 4 113 successful launches

16
16
Ariane 5
370 000 000 $ blown up!

 Unit testing
 Integration testing
 System testing
 …
Ways to Look for Bugs
17

 Unit testing
 System testing
 …
 Dynamic analysis
 Static analysis
Ways to Look for Errors
18

 Unit testing
 System testing
 …
 Dynamic analysis
 Static analysis
Ways to Look for Errors
19

Static analysis is an automatic code review
procedure.
What is Static Analysis?
20

How Does It Help?
22
Static analysis

Oh No!
24
Legacy’s somewhere here!!!

1. Set up continuous analysis
2.
3.
What’s a Good Way to Deal With Old Bugs?
25

2. Protect yourself from new bugs
3.
26

2. Protect yourself from new bugs
3. Gradually go through the analyzer’s report line-
by-line, making sure to spend few resources
each time.
27

How Our Two Developers Fixed 2000 Bugs, That Our
Analyzer Reported in Unreal Engine 4, in 17 Business
Days
28

 My boss found errors in UE 4
and wrote an article
 Developers from Epic Games
liked it a lot
 They wanted to fix more bugs
and trusted that matter to us
How It Started
30

31
Take One
(Or How We Got Our Asses Kicked)

 The most convenient approach: check the
project through Visual Studio
 Luckily, UE has a set of scripts to generate
.vcxproj files
The First Check
32

1.Generate project files
2.Build the project
3.Start the analysis from Visual
Studio
4.???????
5.
The First Analysis Attempt
33

1.Generate project files
2.Build the project
3.Start the analysis from Visual
Studio
4.???????
5.EPIC GAMES FAIL
The First Analysis Attempt
34

 Generated project files are just a wrapper
 These wrappers call the Unreal Build Tool
 Unreal Build Tool starts cl.exe (or clang, for Linux
builds)
How the Unreal Engine Project Is Built
35

 Generated project files are just a wrapper
 These wrappers call the Unreal Build Tool
 Unreal Build Tool starts cl.exe (or clang, for Linux
builds)
 Because of all these layers the analyzer cannot
collect the parameters required for compilation
How Unreal Engine Project Is Built
36

37
Take Two
(Or How We Did Everything Right)

 What if we try to catch compiler calls directly?
The Second Check
38

 What if we try to catch compiler calls directly?
 It’s pretty fortunate that we have a special utility
that can monitor compilation
The Second Check
39

The Second Analysis Attempt
40
1. Start the compilation monitoring
utility before building the project
2. The utility collects all the required
data
3. Right after that’s done, the analysis
starts
4. ???????
5.

1. Start the compilation monitoring
utility before building the project
2. The utility collects all the required
data
3. Right after that’s done,
the analysis starts
4. ???????
5. EPIC WIN!!!
The Second Analysis Attempt
41

Analysis Results
43
 1192 top-level warnings
(Level 1)
 629 second-level warnings
(Level 2)
 1821 warnings in total
(Level 3 not included)

Why Is It Typical to Get Thousands of Warnings?
45
 Notepad++ – 5 241 warnings

46
 WinMerge – 8 786 warnings

47
 Media Player Classic Home Cinema – 33 370 warnings

How to Process Old Errors
49
1. Set up your development process so that it involves
continuous analysis

58
continuous analysis
2. Prevent new errors

2. Prevent New Errors
59
Статанализ

2. Prevent New Errors
60
 How to detect new triggerings among old
warnings?
 Approach 1: incremental analysis
 Approach 2: warning mass suppression (the use
of a suppress base)

How to Work with a Suppress Base
61
 Hide all old warnings
 Get 0 warnings after analysis
 Starting from this moment, all new mistakes
will be detected at once

62

63

64
Old warnings don’t go anywhere!
We just separated them from the new ones and
work on them separately.

Incremental Analysis or a Suppress Base?
65
 When dealing with Unreal Engine 4, we used
only incremental analysis
 Suppress bases are more convenient if you are
the one developing the project
 You can use both approaches simultaneously

66
continuous analysis
2. Prevent new errors
3. Process the analyzer’s report gradually, line-by-line,
and save resources.

3. How to Work with a Huge Report
67
Two approaches:
 The Pareto principle (80/20)
 The ratchet method

The Pareto Principle
69
 In order to fix 80% of errors, reviewing 20% of
warnings is sufficient
 The warnings have already been separated into
these portions and are ready for use

70
 Media Player Classic Home Cinema – 33 370 warnings

71
 33 370 warnings –> 1 935 warnings

20% of warnings in Unreal Engine
72
 1192 top-level warnings
(Level 1)
 629 second-level warnings
(Level 2)
 1821 warnings total
(Level 3 not included)

 The total number of warnings is committed to
the version control system
 Changes are allowed only if they do not add to
this number of errors
The Ratchet Method
74

How the Ratchet Method Works
75

How the Ratchet Method Works
76

 Ivan Ponomaryov — Continuous Static Code
Analysis
A Talk on Related Topic
77

 Expectation
Warning Number Graph
0
5
10
15
20
25
1 2 3 4 5
Warnings
79

 Expectation
Warning Number Graph
80
0
5
10
15
20
25
1 2 3 4 5
Warnings
0
5
10
15
20
25
1 2 3 4 5
Warnings
 Reality

Progress for Unreal Engine 4
81

A Small Dessert
82
 After fixing all errors we found two more:

A Sample Graph for the Ratchet Method
83

 Developers from Epic Games were happy
 They now regularly use static analysis
 As for us… we wrote another article :)
Recap
84

86
Want to Clean Up Legacy Code?
Use Static Analysis!

If the Warnings Count in Thousands:
87
1.Set up your development process so that it involves
continuous analysis
2.Prevent new mistakes
3.Work through the analyzer’s report gradually
(the Pareto and ratchet principles)

Try Static Analysis!
e-mail:
gribkov@viva64.com

Try Static Analysis!
e-mail:
gribkov@viva64.com
Q&A

How to Fix Hundreds of Bugs in Legacy Code and Not Die (Unreal Engine 4)

Recommended

More Related Content

What's hot (20)

Similar to How to Fix Hundreds of Bugs in Legacy Code and Not Die (Unreal Engine 4) (20)

More from Andrey Karpov (20)

Recently uploaded (20)

How to Fix Hundreds of Bugs in Legacy Code and Not Die (Unreal Engine 4)