Hi!
The classic presentation from 2012 is still highly recommended! The APIs were renamed a bit and a couple of bugs were fixed so their example checker doesn’t look exactly the same as it used to, but you’ll still get a great understanding of the basics regardless.
However I’m currently a bit lost on how to approach or even how exactly to create states or transtitions between states.
If I was to guess, a common source of confusion is that you need to start thinking in terms of execution paths. The static analyzer’s path-sensitive engine solves problems that can be formulated as:
“Search for execution paths on which certain events, possibly featuring specific values, happen in a certain order.”
For example, use-after-free can be formulated as “Search for execution paths on which a certain pointer value is freed and then its pointee is accessed”.
If your problem can’t be formulated this way, you’re probably not using the right tool! If your problem can be formulated this way, then your checker callbacks are the events you want to see on the path, and your state records the information about which events have already happened on the current path.
For example, if you want to catch use-after-free, you subscribe to the “free” event (say, checkPostCall for function free()) and to the “use” event (say, checkLocation to react on all reads and writes in memory). At “free” you record pointers that were freed in the state. At “use” you warn if the pointer through which the read is performed is recorded in the state.
Note that the checker doesn’t need to handle how the pointer value travels between variables. The engine handles that automatically! - by the virtue of knowing the values of all variables.
The static analyzer explores the execution paths automatically, and lets checkers react on events that happen. The order of path exploration can be quite arbitrary! It may explore one path for a bit, then explore another path for a bit, then come back to the old path. This is why it’s important to record things in the State - that’s the only way to correctly and efficiently keep track of things you know about the current path! It also makes it difficult to debug things with print-statemets: it’s hard to figure out from a print which execution path is currently explored. So the best way to debug the static analyzer is to print the entire graph of explored paths (so-called ExplodedGraph), and print your custom section of the state as part of it.
There’s a more recent, more in-depth talk. You probably don’t need any of that, but closer to the end I demonstrate some debugging techniques, like how to print the graph and how to read it.
There’s also my old workbook (https://github.com/haoNoQ/clang-analyzer-guide/releases/download/v0.1/clang-analyzer-guide-v0.1.pdf) which may help you navigate the callbacks and understand symbolic values a bit better.
I even might have to deal with an if statement where one of the functions is called.
checkBranchCondition may help!