I'm testing a crypto library under the sanitizers. Crypto libraries
are notorious for doing clever (and sometimes undefined) things to
integers.
I'm catching a number of errors similar to below. Is (or should)
unsigned wrap be an error condition? I thought it was implementation
defined. Or is -fsanitize=integer catching signed integer overflow
(which is undefined behavior) and incorrectly reporting it?
Related: is -fsanize-integer Peng and Regehr's Integer Overflow
Checker? I'm very interested in IOC because Intel's ICC is ruthless
about removing code with undefined behavior.
Jeff
$ ./testlib
...
./crypt/sha1locl.h:255:2: runtime error: unsigned integer overflow:
11602415833421178322 + 16313194667034588172 cannot be represented in
type 'unsigned long'
./crypt/sha1locl.h:256:2: runtime error: unsigned integer overflow:
13854689891680008357 + 7855820476526050265 cannot be represented in
type 'unsigned long'
./crypt/sha1locl.h:257:2: runtime error: unsigned integer overflow:
7089814647387623028 + 17503132371316721671 cannot be represented in
type 'unsigned long'
./crypt/sha1locl.h:257:2: runtime error: unsigned integer overflow:
6146202944994793083 + 12350673664601905734 cannot be represented in
type 'unsigned long'
./crypt/sha1locl.h:259:2: runtime error: unsigned integer overflow:
17543271277370323423 + 5826006124154911379 cannot be represented in
type 'unsigned long'
./crypt/sha1locl.h:261:2: runtime error: unsigned integer overflow:
15261508094756360537 + 16576356271601945904 cannot be represented in
type 'unsigned long'
./crypt/sha1locl.h:263:2: runtime error: unsigned integer overflow:
7467814761072524174 + 11287875045752437900 cannot be represented in
type 'unsigned long'
I'm testing a crypto library under the sanitizers. Crypto libraries
are notorious for doing clever (and sometimes undefined) things to
integers.
I'm catching a number of errors similar to below. Is (or should)
unsigned wrap be an error condition? I thought it was implementation
defined. Or is -fsanitize=integer catching signed integer overflow
(which is undefined behavior) and incorrectly reporting it?
Notice for "-fsanitize=integer" they say "Enables checks for undefined or suspicious integer behavior." The 'suspicious' part refers to unsigned overflow, which, though well-defined, is often unexpected (though probably not in a crypto lib).
I think you want -fsanitize=undefined.
Related: is -fsanize-integer Peng and Regehr's Integer Overflow
Checker? I'm very interested in IOC because Intel's ICC is ruthless
about removing code with undefined behavior.
Right. Specifically, the class of security bugs where the calculation of anallocation size overflows overwhelmingly involves unsigned overflow.
But of course there are also many reasonable calculations, like hash
functions, that should ignore unsigned overflow. So it’s important for
the user to provide direction here.
Actually, I would instead say that -fsanitize=integer is the spiritual successor of IOC. It is Regehr and Peng’s idea, it is not the IOC implementation of that idea, but some of the implementation is due to their group. Here’s the story:
-fsanitize=undefined (“UBSan”) was inspired by IOC. With John Regehr’s consent, I started with the intention of committing the IOC patches into upstream Clang. However, the scope of the project got extended to an attempt to catch a much more comprehensive set of undefined behavior, and what I ended with was a near-complete rewrite – there is essentially no code in common between UBSan and IOC, and UBSan has a large number of other sanitizers that were never part of IOC. -fsanitize=unsigned-integer-overflow is not part of UBSan, since it is not a check for undefined behavior.
-fsanitize=integer is -fsanitize=undefined’s checks for integer undefined behavior, plus -fsanitize=unsigned-integer-overflow, and its feature set is similar to that of IOC (but not exactly the same). -fsanitize=unsigned-integer-overflow was the work of Will Dietz, under (I believe) the direction of John Regehr, and Will also contributed various fixes and features to the rest of UBSan (in some cases, adding features that were present in IOC but not in UBSan).