Pre-LLVM-DEV'23 -- Embedded Toolchains Workshop

Similarly to the post Pre-LLVM-DEV'23 --- GPU/Offloading Workshop about the potential of a GPU/Offloading Workshop at the 2023 LLVM Developer’s meeting. We’re canvassing opinion to see if there is enough interest in an embedded toolchains workshop.

We (Arm) can present some material and offer up some items for discussion, but it would need additional content, and enough people participating to make it worth more than a roundtable. In particular topics that require code-review and design discussion would be particularly useful to address. Some examples:

  • LTO in linker scripts
  • MC DC code coverage
  • Bare-metal profiling
  • Code-size and performance
  • Embedded linker features and diagnostics
  • Using LLVM libc in embedded features.

We can discuss this in the next LLVM Embedded Toolchains Working Group call on Thursday the 22nd June. Feel free to reply to this thread if you are interested?

Original post asking for proposals: Call for Workshops at the 2023 LLVM Developers' Meeting

1 Like

We (Fuchsia) would be interested in participating and helping with the organization in this workshop. We would be also happy to present our progress in this space.

Your list of topics matches our TODO list for embedded development and it would be great to have time to dive into these topics in more detail, beyond what’s typically possible in a roundtable.

To make this workshop worthwhile, I think it would be paramount to have owners of different areas attend; off the top of my head I can think of @maskray for LLD, @snidertm and @ilovepi for LTO and linker scripts, @barrelshifter and @ellishg for coverage, @evodius96 for MC/DC coverage, @sivachandra and @michaelrj-google for LLVM libc, @ldionne for libc++, and likely more I forgot.

1 Like

+1

I’m interested from the perspective of GPU offloading.
I think we face similar problems, to some degree at least, and for things like libc allocators, we might want to walk in lock step.

I’d be interested. We do various things in libc++ to support embedded, for example we have generic ways to carve out parts of the library that are typically not available on embedded systems. I think this is an important area for C++ and worth discussing.

I am interested from the libc side.

Thanks for the replies everyone. I think there is enough interest to submit the proposal. I’ll do that within the next few working days.

1 Like

I’d be interested in attending. This might be a dumb question, but does embedded include mobile? As far as I understand, we do have similar goals like codesize, but resources like CPU and memory aren’t quite as constrained. CC @kyulee-com who might also be interested in attending.

1 Like

It is mostly going to be smaller systems than userspace applications on mobile, although it could encompass firmware development. As you point out there is an overlap of concerns such as code-size. The areas that wouldn’t be as interesting are likely to be: MC DC code coverage, bare-metal profiling and embedded linker features - although linker diagnostics may still be useful.

I’m filling out the form to submit the proposal today. I looks like I need to nominate a second organizer. @petrhosek are you OK if I can put your name down as the second organizer for the nomination? I’m expecting we can change that at a later time.

I’m suggesting the following for the description of the event. Will be sending this out towards 6:00pm UK time. Happy to incorporate any feedback.

I’m proposing half a day and < 25 attendees.

Short 1 paragraph description that describes to attendees what they can expect by attending.
Embedded systems software, which for the purposes of the workshop we’ll define as ranging from a no-operating system microcontroller project, to embedded linux, has some requirements that are different from userspace software that runs on top of an OS. The purpose of the workshop is to focus on the embedded specific requirements and coming up with an approach for addressing these in the open-source llvm-project. The focus will be on sharing knowledge of downstream solutions and discussing requirements and design of upstream features rather than a tutorial for how to build a toolchain for embedded systems.

Detailed description or agenda of the workshop
We haven’t got a detailed agenda confirmed. Our proposed list of topics

Using LTO for embedded development.
LTO is seen as a strength of the llvm-project, many of which should benefit embedded systems. However getting it working on an embedded project is too complex for many users, particularly when linker scripts are involved. There have been two LLVM dev meeting presentations on downstream solutions for how to interact with linker scripts 2022 LLVM Dev Mtg: Link-Time Attributes for LTO: Incorporating linker knowledge into the LTO... - YouTube and 2017 LLVM Developers’ Meeting: T. Edler Von Koch “Bringing link-time optimization to the ... ” - YouTube . We would like to agree on a design for getting this upstream.

Instrumentation
The existing implementation of the profiling runtime in compiler-rt won’t build for many embedded systems due to dependencies on the OS. There is scope for a stripped down runtime that can build for an embedded system. To get to this point some refactoring will be needed.

Embedded systems are often used in projects with functional safety requirements. A particular form of code-coverage called MC-DC is required for some safety integrity levels. There is a patch set in upstream review, if that hasn’t landed by the time of the workshop it can be discussed.

Optimizing for code size
When resources are limited small code size is often more important than highest performance. For the purposes of the workshop we are likely to concentrate on higher-level features such as the outliner than individual back-end optimizations.

Much recent work has gone into thin-LTO and performance, whereas embedded projects tend to be small enough to make use out of traditional whole program and the size reduction features that LTO offers without the scaling problems. Finding a way forward for a LTO for smallest code-size is a topic for discussion.

This topic has some overlap with the attempts to lower code-size for mobile applications, although the features used for mobile development may require code-generation work and that may not be available on embedded targets.

Embedded Linker features and diagnostics
Embedded systems often have complex memory maps which require complex linker scripts that are easy to get wrong. Many proprietary toolchains have additional features such as data compression, advanced overlay support, and automatic distribution of sections across memory regions. It would be great to get these features implemented in upstream linkers such as LLD and GNU ld. The major blocker is often justification for the additional implementation complexity.

Extra complex linker scripts are often hard to debug. There is scope for additional tracing and diagnostics, particularly machine readable output or GNU compatible output.

LLVM libc and libc++ for embedded systems
There aren’t many open-source C libraries suitable for embedded systems with newlib and lately picolibc as the main choices. LLVM libc would like to scale down to embedded systems. We would like to discuss experiences developers have with LLVM libc and gather requirements, such as the interface to heap allocation, that LLVM libc would need to be the default choice for embedded toolchains based on LLVM.

C++ use in embedded systems is on the rise, particulary in the automotive area with the advent of Autosar Adaptive. A large fraction of the C++ standard library as implemented by libc++ can be used by embdedded systems by excluding areas that depend on OS support such as the filesystem. Many limited file IO can be done, such as using the serial port or semihosting. There is an opportunity to come up with a shared view of what libc++ and libc++abi configuration options would be useful to support, and how we can provide buildbot support for them.

1 Like

Request now submitted. I’m sure there will be an opportunity to edit the details later.

I know there’s already been some thought concerning baremetal code coverage solutions.
At the 2020 LLVM Developers’ Meeting, I gave a lightning talk on embedded code coverage use cases that highlighted removal of runtime components (unneeded in a baremetal context) to reduce the runtime code size footprint. These are things that we’ve done in our downstream compiler at TI, but I suspect there may be ways to introduce build configurations for upstream that might be useful. Perhaps we could discuss this during the workshop.

Yes that sounds good. I think we could cover that in the instrumentation section. My apologies if I forgot anything in the detailed description. I’m sure we’ll be able to edit it later.

Just a quick note to mention that the workshop has been accepted and will be a half day event on the 10th October.

I’ll be thinking about how best to organise the workshop and will make a separate post about that soon.

1 Like