Why this is needed
VxWorks is a very popular real-time operating system and has many users across multiple industries. It has been undergoing active development with more features added and has 2-3 releases each year. Since its latest release 22.03, VxWorks starts to fully support LLVM AddressSanitizer. We believe that a lot of VxWorks users will benefit from that. But current AddressSanitize actually doesn’t support VxWorks OS, all VxWorks specific changes are made in terms of private patches. Each time user builds VxWorks, a copy of AddressSanitizer code is downloaded and then those patches are applied to it. This approach is very inconvenient. The patch applying can be easily failed if there are some new major changes made in the latest AddressSanitizer. If those patches can be up-streamed to the main stream, then no patches are needed and that will bring a lot of value.
Upstream strategy
No two operating systems are completely same. AddressSanitizer heavily relies on OS specific macro to reuse the implementation, such as SANITIZER_LINUX/SANITIZER_FUCHSIA/SANITIZER_SOLARIS. Therefore, VxWorks has to define its own macro SANITIZER_VXWORKS too in order to reuse existing implementation. This kind of change couldn’t be avoided. For example,
#include "sanitizer_common/sanitizer_platform.h"
#if SANITIZER_FREEBSD || SANITIZER_FUCHSIA || SANITIZER_LINUX || \
SANITIZER_NETBSD || SANITIZER_SOLARIS **|| SANITIZER_VXWORKS**
Besides reusing, there are some implementation specific to VxWorks. For example, some API’s name are different, some API may be not supported, or some data type are defined differently. Not all these changes should be up-streamed otherwise it would cause great maintenance difficulty for other developers. To reduce this kind of change in main stream, VxWorks on one side should adopt common implementation such as POSIX API as many as possible. On the other hand, some quite specific implementation should be still kept in terms of private patches.
All the changes to be up-streamed are guarded by the new macro SANITIZER_VXWORKS and no other code will be broken. And we keep the occurrences of this macro as few as possible. In this way, the final upstream will not bring extra maintenance workload in future.
Maintenance Plan
Our VxWorks specific changes are guarded by SANITIZER_VXWORKS and will not break any other OS. There is only possibility that VxWorks is broken by new code in main stream. But that is not an issue since VxWorks will regularly rebase the main stream code and fix regression by ourselves and then properly provide feedback to the main stream.
Code Review
The initial code review has been launched at ⚙ D117790 [AddressSanitizer] Add AddressSanitizer support for VxWorks OS. Any comments are appreciated.