The analyzer implementation works on the gimple-SSA representation. (I chose this in the hopes of making it easy to work with LTO to do whole-program analysis).
The implementation is read-only: it doesn’t attempt to change anything, just emit warnings.
The gimple representation can be seen using -fdump-ipa-analyzer.
Tip: If the analyzer ICEs before this is written out, one workaround is to use --param=analyzer-bb-explosion-factor=0 to force the analyzer to bail out after analyzing the first basic block.
First, we build a supergraph which combines the callgraph and all
of the CFGs into a single directed graph, with both interprocedural and
intraprocedural edges. The nodes and edges in the supergraph are called
“supernodes” and “superedges”, and often referred to in code as
snodes and sedges. Basic blocks in the CFGs are split at
interprocedural calls, so there can be more than one supernode per
basic block. Most statements will be in just one supernode, but a call
statement can appear in two supernodes: at the end of one for the call,
and again at the start of another for the return.
The supergraph can be seen using -fdump-analyzer-supergraph.
We then build an analysis_plan which walks the callgraph to
determine which calls might be suitable for being summarized (rather
than fully explored) and thus in what order to explore the functions.
Next is the heart of the analyzer: we use a worklist to explore state within the supergraph, building an "exploded graph". Nodes in the exploded graph correspond to <point, state> pairs, as in "Precise Interprocedural Dataflow Analysis via Graph Reachability" (Thomas Reps, Susan Horwitz and Mooly Sagiv).
We reuse nodes for <point, state> pairs we’ve already seen, and avoid tracking state too closely, so that (hopefully) we rapidly converge on a final exploded graph, and terminate the analysis. We also bail out if the number of exploded <end-of-basic-block, state> nodes gets larger than a particular multiple of the total number of basic blocks (to ensure termination in the face of pathological state-explosion cases, or bugs). We also stop exploring a point once we hit a limit of states for that point.
We can identify problems directly when processing a <point, state> instance. For example, if we’re finding the successors of
<point: before-stmt: "free (ptr);",
state: {"ptr": freed}>
then we can detect a double-free of "ptr". We can then emit a path to reach the problem by finding the simplest route through the graph.
Program points in the analysis are much more fine-grained than in the CFG and supergraph, with points (and thus potentially exploded nodes) for various events, including before individual statements. By default the exploded graph merges multiple consecutive statements in a supernode into one exploded edge to minimize the size of the exploded graph. This can be suppressed via -fanalyzer-fine-grained. The fine-grained approach seems to make things simpler and more debuggable that other approaches I tried, in that each point is responsible for one thing.
Program points in the analysis also have a "call string" identifying the
stack of callsites below them, so that paths in the exploded graph
correspond to interprocedurally valid paths: we always return to the
correct call site, propagating state information accordingly.
We avoid infinite recursion by stopping the analysis if a callsite
appears more than analyzer-max-recursion-depth in a callstring
(defaulting to 2).
Nodes and edges in the exploded graph are called “exploded nodes” and
“exploded edges” and often referred to in the code as
enodes and eedges (especially when distinguishing them
from the snodes and sedges in the supergraph).
Each graph numbers its nodes, giving unique identifiers - supernodes are referred to throughout dumps in the form ‘SN': index’ and exploded nodes in the form ‘EN: index’ (e.g. ‘SN: 2’ and ‘EN:29’).
The supergraph can be seen using -fdump-analyzer-supergraph-graph.
The exploded graph can be seen using -fdump-analyzer-exploded-graph and other dump options. Exploded nodes are color-coded in the .dot output based on state-machine states to make it easier to see state changes at a glance.
There’s a tension between:
For example, in general, given this CFG:
A
/ \
B C
\ /
D
/ \
E F
\ /
G
we want to avoid differences in state-tracking in B and C from leading to blow-up. If we don’t prevent state blowup, we end up with exponential growth of the exploded graph like this:
1:A
/ \
/ \
/ \
2:B 3:C
| |
4:D 5:D (2 exploded nodes for D)
/ \ / \
6:E 7:F 8:E 9:F
| | | |
10:G 11:G 12:G 13:G (4 exploded nodes for G)
Similar issues arise with loops.
To prevent this, we follow various approaches:
We avoid merging pairs of states that have state-machine differences, as these are the kinds of differences that are likely to be most interesting. So, for example, given:
if (condition)
ptr = malloc (size);
else
ptr = local_buf;
.... do things with 'ptr'
if (condition)
free (ptr);
...etc
then we end up with an exploded graph that looks like this:
if (condition)
/ T \ F
--------- ----------
/ \
ptr = malloc (size) ptr = local_buf
| |
copy of copy of
"do things with 'ptr'" "do things with 'ptr'"
with ptr: heap-allocated with ptr: stack-allocated
| |
if (condition) if (condition)
| known to be T | known to be F
free (ptr); |
\ /
-----------------------------
| ('ptr' is pruned, so states can be merged)
etc
where some duplication has occurred, but only for the places where the the different paths are worth exploringly separately.
Merging can be disabled via -fno-analyzer-state-merge.
Part of the state stored at a exploded_node is a region_model.
This is an implementation of the region-based ternary model described in
"A Memory Model for Static Analysis of C Programs"
(Zhongxing Xu, Ted Kremenek, and Jian Zhang).
A region_model encapsulates a representation of the state of
memory, with a store recording a binding between region
instances, to svalue instances. The bindings are organized into
clusters, where regions accessible via well-defined pointer arithmetic
are in the same cluster. The representation is graph-like because values
can be pointers to regions. It also stores a constraint_manager,
capturing relationships between the values.
Because each node in the exploded_graph has a region_model,
and each of the latter is graph-like, the exploded_graph is in some
ways a graph of graphs.
Here’s an example of printing a program_state, showing the
region_model within it, along with state for the malloc
state machine.
(gdb) call debug (*this)
rmodel:
stack depth: 1
frame (index 0): frame: ‘test’@1
clusters within frame: ‘test’@1
cluster for: ptr_3: &HEAP_ALLOCATED_REGION(12)
m_called_unknown_fn: FALSE
constraint_manager:
equiv classes:
constraints:
malloc:
0x2e89590: &HEAP_ALLOCATED_REGION(12): unchecked ('ptr_3')
This is the state at the point of returning from calls_malloc back
to test in the following:
void *
calls_malloc (void)
{
void *result = malloc (1024);
return result;
}
void test (void)
{
void *ptr = calls_malloc ();
/* etc. */
}
Within the store, there is the cluster for ptr_3 within the frame
for test, where the whole cluster is bound to a pointer value,
pointing at HEAP_ALLOCATED_REGION(12). Additionally, this pointer
has the unchecked state for the malloc state machine
indicating it hasn’t yet been checked against NULL since the allocation
call.
We need to explain to the user what the problem is, and to persuade them
that there really is a problem. Hence having a diagnostic_path
isn’t just an incidental detail of the analyzer; it’s required.
Paths ought to be:
Without state-merging, all paths in the exploded graph are feasible (in terms of constraints being satisfied). With state-merging, paths in the exploded graph can be infeasible.
We collate warnings and only emit them for the simplest path e.g. for a bug in a utility function, with lots of routes to calling it, we only emit the simplest path (which could be intraprocedural, if it can be reproduced without a caller).
We thus want to find the shortest feasible path through the exploded graph from the origin to the exploded node at which the diagnostic was saved. Unfortunately, if we simply find the shortest such path and check if it’s feasible we might falsely reject the diagnostic, as there might be a longer path that is feasible. Examples include the cases where the diagnostic requires us to go at least once around a loop for a later condition to be satisfied, or where for a later condition to be satisfied we need to enter a suite of code that the simpler path skips.
We attempt to find the shortest feasible path to each diagnostic by first constructing a “trimmed graph” from the exploded graph, containing only those nodes and edges from which there are paths to the target node, and using Dijkstra’s algorithm to order the trimmed nodes by minimal distance to the target.
We then use a worklist to iteratively build a “feasible graph” (actually a tree), capturing the pertinent state along each path, in which every path to a “feasible node” is feasible by construction, restricting ourselves to the trimmed graph to ensure we stay on target, and ordering the worklist so that the first feasible path we find to the target node is the shortest possible path. Hence we start by trying the shortest possible path, but if that fails, we explore progressively longer paths, eventually trying iterations through loops. The exploration is captured in the feasible_graph, which can be dumped as a .dot file via -fdump-analyzer-feasibility to visualize the exploration. The indices of the feasible nodes show the order in which they were created. We effectively explore the tree of feasible paths in order of shortest path until we either find a feasible path to the target node, or hit a limit and give up.
This is something of a brute-force approach, but the trimmed graph hopefully keeps the complexity manageable.
This algorithm can be disabled (for debugging purposes) via -fno-analyzer-feasibility, which simply uses the shortest path, and notes if it is infeasible.
The above gives us a shortest feasible exploded_path through the
exploded_graph (a list of exploded_edge *). We use this
exploded_path to build a diagnostic_path (a list of
events for the diagnostic subsystem) - specifically a
checker_path.
Having built the checker_path, we prune it to try to eliminate
events that aren’t relevant, to minimize how much the user has to read.
After pruning, we notify each event in the path of its ID and record the
IDs of interesting events, allowing for events to refer to other events
in their descriptions. The pending_diagnostic class has various
vfuncs to support emitting more precise descriptions, so that e.g.
returning possibly-NULL pointer to 'make_obj' from 'allocator'
for a return_event to make it clearer how the unchecked value moves
from callee back to caller
second 'free' here; first 'free' was at (3)
and a use-after-free might use
use after 'free' here; memory was freed at (2)
At this point we can emit the diagnostic.