Proposal: Static Analysis Ergonomics
- to make it easier to run static analysis tools during the development of code and of patches
- to provide *mechanisms* to allow distributions such as Fedora to set *policy* about what static analysis tools are run during a build, and which results are expected to gate a build. For example, I want a distribution to be able to create e.g. a "network facing daemon" policy file that describes what analysis should be run, and be able to have a one-liner in an rpm .spec file/.deb file that enforces that policy, -Werror, style (with the ability to override it)
I'm looking at adding new functionality to gcc 8 upstream, and this proposal is intended as an RFC to check that I'm on the right track, and to gather ideas.
The problem and the proposed solution
- IMHO static analysis is done too late, if at all: static analysis tools are run as an optional extra, "on the side", rather than in developers' normal workflow. Analysis results are reviewed (if at all) in some kind of review tool, rather than when the code is being edited, or patches being prepared. It's a major PITA to be told via a logfile that tool $FOO reports a problem in bar.c line 1150, 3 hours after I edited that code; better to tell the developer immediately during the initial compile when they're right there in the editor.
- better to have an easy way to run analyzer(s) as development happens, as part of the edit-compile-test cycle - problems are reported immediately, and can be acted on immediately
- hence the proposal is to give gcc a way to integrate invoking 3rd-party static analysis tools, emitting the results through gcc's diagnostic subsystem as if they were plugins
- also the ability to run analyzer(s) when packages are built. Have the ability for distributions to create a variety of pre-canned policies for analyzers (e.g. "network-facing daemons", "suid binaries", etc)
- metadata tagging: all binaries to be tagged with information on what analyzers were run, what options etc (and also, which compiler was run, and what options). Potentially have "dump_file" information from optimization passes stored in the metadata also. Have a tool to query all of this, so that we can do a distro-wide query: "tell me all suid binaries that weren't compiled with such-and-such warnings enabled and at -Werror".
- Can/should we break the build if there are issues?
- Yes: but have a way to opt-in easily: if the tool is well-integrated with the compiler: e.g.
then upstream developers and packagers can turn on the setting, and see what breaks, and fix it naturally within an compile-edit-test cycle
- Does this slow the build down?
- Yes: but you can choose which analyzers run, and can choose to turn them off. It ought to parallelize well. I believe users will prefer to turn them on, and have Koji builders burn up the extra CPU cycles.
- upstream developer
- distribution packager (Fedora, Debian)
- Fedora security team (setting policy etc)
- setting policy
- after a vulnerability, looking to assess the scope; could we have caught this earlier?
* clang-analyzer * cppcheck * findbugs * cpychecker??? * anything else?
A possible new option in GCC 8:
-frun-analyzers=/usr/share/analyzers/userspace/network-facing-service -frun-analyzers=/usr/share/analyzers/userspace/suid-binary -frun-analyzers=/usr/share/analyzers/userspace/application -frun-analyzers=/usr/share/analyzers/userspace/default -frun-analyzers=/usr/share/analyzers/kernel
Alternatively, these would simply be gcc "command" files via
Idea is to provide mechanism, and for the distribution to decide on some standard policies.
$ rpm -qf /usr/share/analyzers/userspace/network-facing-service analysis-policy-userspace-network-facing-service-0.1-1.noarch.fc27
hence would have:
which in turn would
Require the analyzers themselves.
Look at Firehose:
and look at mock-with-analysis:
for watermarking binaries with metadata.
Do we run the risk of breaking "configure" tests?
Proof-of-concept patch kit for gcc: RFC: integrated 3rd-party static analysis support