15 February 2010

Static analysis: a case study of Coverity

The Communications of the ACM has a long but interesting article about the experiences of the Coverity team and the development of the Coverity Static Analysis product.

(The article has lots of war stories. I enjoyed reading it.)

There are many interesting technical hurdles the Coverity team has had to deal with (e.g. standards vs. reality in language implementations; integrating into complicated and established build systems) in order to make Coverity Static Analysis find as many bugs as possible in user code. But the Coverity team also recognized that in order to make a sale or keep a customer, the output of Coverity Static Analysis must be simple, consistent, and easy to understand— objectives that are often at odds with the goal of finding as many bugs as possible.

For example, one point of tension is that every analysis that identifies a bug has to be able to explain to the user the exact circumstances under which that bug could manifest itself, and

a not-understood bug report is commonly labeled a false positive, rather than spurring the programmer to delve deeper. The result? We have completely abandoned some analyses that might generate difficult-to-understand reports.

No comments:

Post a Comment