[+cfe-dev, since I think this might interest some Clang developers/C++
standards sticklers]
> 5. The C++ standard only defines relational operators for pointers to
> elements of the same array, so arena::pointer_in_buffer() triggers
> undefined behavior whenever it's used to test a pointer allocated by
> malloc(). The best solution to this that I know of is to do the
> comparisons on integers instead, and hope that the compiler does
> implement an "unsurprising" pointer-to-integer mapping as encouraged
> (but not required) by the standard.It isn't undefined behavior if I use std::less ([comparisons]/p14),
and I'm taking advantage of the fact that on all platforms being
targeted, std::less<T*> is implemented with nothing more complicated
than pointer comparison.
Hm, true, the standard does guarantee std::less imposes a total
ordering on pointers, though I don't believe that forbids orderings
that interleave pointers to distinct arrays. E.g., it seems perfectly
standards conforming (but "surprising") for this program to output 1:
#include <functional>
#include <iostream>
int main() {
std::less<char *> lt;
char x, y;
std::cout << (lt(&x, &y) && lt(&y, &x + 1)) << std::endl;
}
So I think this actually reveals a bug in libc++'s implementation of
std::less, etc. They need partial specializations for pointer types
to properly guarantee a total ordering, otherwise the above code
triggers undefined behavior with libc++'s current definitions due to
the pointer comparisons between &y and &x (and between &y and &x + 1).
We should then also utilize std::less_equal in cxa_demangle.cpp
instead of directly comparing pointers. If we wanted to go further,
in arena::allocate() we could assert that the pointer returned by
std::malloc() does not satisfy pointer_in_buffer().
Index: libcxx/include/__functional_base