Background
In embedded systems, it’s common for different parts of the memory to have different performance or power characteristics. These systems also typically don’t have an MMU and use a single address space, and usually rely on linker scripts to map the code and the data to appropriate locations.
To achieve this, at the C/C++ level, the applications typically utilize the section attribute:
__attribute__((section(".special"))) int buffer[1024];
At the linker script level, they would then match the section and place it in an appropriate memory region:
MEMORY {
SPECIAL(rwxi) : ORIGIN = 0x00000000, LENGTH = 64k
...
}
SECTIONS {
.special : { *(.special); } > SPECIAL
...
}
While this approach achieves the desired goal, it has several shortcomings.
First of all, it breaks the linker GC (that is --gc-sections). When the -ffunctions-sections and -fdata-sections options are used, the compiler places each symbol into its own section, and the linker can then discard unused sections by tracing the relocations.
When we use the section attribute, all symbols are placed into a single section and as such cannot be discarded individually.
To work around this issue, we can manually replicate the compiler behavior and use a separate section for each symbol:
__attribute__((section(".special.buffer"))) int buffer[1024];
This approach is used for example by Pico SDK.
For zero-initialized (explicitly or implicitly) variables, compilers typically use a separate .bss section to reduce binary size since these variables can be allocated during initialization.
We can achieve the same effect by again having a dedicated section which is an approach also used by Pico SDK, but this further increases the code complexity and adds maintenance overhead since these attributes need to be kept up-to-date during code changes.
Proposal
What developers are really trying to achieve is annotating certain symbols as living in a special part of the memory without changing the existing rules for section allocation and placement. To support that, our proposal is to introduce support for memory regions as a new memory attribute:
__attribute__((memory("special"))) int buffer[1024];
In LLVM IR this would be represented as:
@buffer = dso_local global [1024 x i32] zeroinitializer, memory "special", align 16
To represent the mapping of symbols to regions at the object file level, we propose introducing a new custom section type SHT_LLVM_MEMORY. This section contains an array of fixed-size entries where each entry is 2 uint32_t integers: section number and string table offset, where the string table offset points to the region name. sh_link should point to the particular string table section, as it does in SHT_SYMTAB sections.
To support the use of address spaces from linker scripts, we propose introducing a new matcher INPUT_SECTION_MEMORY (akin to INPUT_SECTION_FLAGS):
SECTIONS {
.special : { INPUT_SECTION_MEMORY(special) *(.text .text.*) } > SPECIAL
.text : { INPUT_SECTION_MEMORY(!special) *(.text .text.*) } > RAM
}
The matcher would use exact match (plus negation), but could be extended to support fnmatch (plus negation) pattern as for sections if there’s a valid use case.
Initially, we plan to implement this feature only for the ELF format which is most commonly used in embedded systems, but there’s no fundamental reason why this idea couldn’t be extended to other object formats in the future.
We are open to suggestions for alternative names as memory may be overly generic. Some of the other ideas we considered were memory_name or section_category.
Alternatives
Instead of introducing a new attribute and LLVM IR construct, we could reuse address_space, this has some downsides though. Most notably, address_space has type system implications—pointers to the same type in different address spaces are incompatible—as well as target-dependent semantics which may be undesirable. Furthermore, address_spaces are identified by an integer which affects usability. Support for string keys for address space has already been proposed before but there hasn’t been any progress in implementing this proposal and it would likely require significant changes throughout Clang and LLVM.
AMD also described the concept of memory_spaces as part of the proposed DWARF Extensions For Heterogeneous Debugging. These appear to have the same constraints as address_spaces though.