LLVM Community Testimonial on Open Access

Below is the testimonial from the LLVM community on the importance of open access to ISO documents.

The LLVM Foundation is a US-based 501(c)(3) nonprofit organization which exists to support the LLVM Project, an open source project which implements support for ISO standards such as ISO/IEC 9899, ISO/IEC 14882, and ISO/IEC 1539 through compilers like Clang and Flang. Clang was initially released in 2007, is freely available, and is one of the most widely deployed compilers for C and C++ in the world with an estimated user-base counted in the millions. Clang is also unique in that it is permissively licensed to encourage others to use Clang as the basis for their own implementations of these standards. As such, there are dozens of Clang-derived compilers (commercial or otherwise) which inherit the work done by the open source community. The LLVM Project averages about 1000 unique contributors per month, has over 17,000 downstream forks, is purely volunteer-based, and has no organizational membership requirements. Contributors can be anyone 13 years old or older and live in any geographical location. Not all contributors are members of organizations which participate in ISO.

The community has several kinds of developers who need access to standards documents: authors who are working to implement a new feature of a programming language, reviewers who are reviewing that work to ensure it is correct, and issue triagers trying to determine whether a particular outcome conforms to the standard or not. The first two groups utilize feature proposal documents and their revisions to understand the basic shape of the feature and how it evolved. They also use the final standard to ensure correct conformance in case of changes late in standardization that isn’t tracked via a paper, such as resolutions to national body comments. Once the feature is fully implemented, the feature entry on the public status tracking page is updated so that users know when features began being supported by the implementation. A similar process is used for addressing defect reports or other issues from the standard. Issue triagers generally can work from either a working draft if the feature is still being actively standardized or from the final standard otherwise.

Losing open access to documents would effectively prevent the LLVM Project from implementing future ISO standards. The programming language standards documents we implement are significant, complex bodies of work. For example, ISO/IEC 14882:2024 is over 2100 pages, not including other standards needed as references. Our users expect us to support each revision of the standard via a configuration option so that they can select which standard is in effect. As a result, our implementation supports e.g., ISO/IEC 14882:2024, ISO/IEC 14882:2020, ISO/IEC 14882:2017, ISO/IEC 14882:2014, and so on. It would be prohibitively expensive to implement the standard from scratch on every revision; instead, we incrementally build on the work done in support of the previous standard. This work is often done while the next revision of the standard is still in development so that we can provide feedback to the standards committee before a feature is finalized. This is why efforts cannot start from the final published standard; it is impractical to determine which changes from the previous revision are related to any given feature and we lose the ability to provide critical feedback to the committees by waiting until the standard is published. The only practical way in which to implement such complex standards is on a feature-by-feature basis, which means we need access to feature proposals, working drafts, and editor reports. The feature proposals help the community to understand the intent of a given feature, the editor’s reports tell us which features have been fully applied to a working draft, and the working drafts help mitigate the risks involved from complex interactions between features.

Additionally, losing open access to documents introduces difficulties with recruiting and training future contributors. Our community collaborates with new developers who gain interest and valuable experience through their existing access to these documents, which is how we replace developers lost to attrition. The financial barriers to regain access will be a hindrance, particularly for students and recent college graduates, which risks our ability to recruit new talent to continue supporting the standards long-term.

Further, open access to standards documents is necessary for communicating with our users. Users file issues which require thorough investigation of the feature to better understand whether we correctly implement it or not. When collaborating with users on issue reports, we often cite information found in feature proposals and working drafts to justify particular behaviors of the implementation. Losing public access to these documents means that citations to the standard are likely not verifiable by our users, so they cannot validate or refute our interpretation of the standard.

Unfortunately, the closed model for participation under ISO does not align well with an open, highly-distributed community like ours. While the LLVM Foundation has a corporate organizational structure and could join a National Body to provide community members a path to access these documents, it is impractical for the LLVM Foundation to do so. The LLVM Project itself is where all the work happens to implement the standards and those members come and go as they have time and interest in volunteering. The LLVM Foundation is a nonprofit with fewer than three paid employees, so the overhead of trying to manage access for a thousand developers is a significant, unaffordable burden. The LLVM Project does not require contributors to provide personally identifiable information (not even their legal name) and so the project does not turn away contributions due to the age of the contributor, where they live, etc. Any sort of membership tracking would require the LLVM Foundation to retain personally identifiable information that needs to be protected, and is subject to GDPR, CCPA, and other laws which are additional burdens. Finally, there are difficulties such as whether minors can enter into binding membership agreements in all jurisdictions because the community allows contributions regardless of age or location.

Our community greatly appreciates the open access we’ve had for these documents since we began our efforts almost 20 years ago. Our project would not exist without open access and we hope ISO and IEC will not remove access to such critical documents moving forward.

Tanya Lattner
Executive Director & President, LLVM Foundation

Aaron Ballman
Clang Lead Maintainer, Clang Area Team Chair, LLVM Project Council Chair

23 Likes