This is a proposal to add an option to lld that forces .dynamic sections to be read-only.
The .dynamic section is almost read-only except for the DT_DEBUG entry which requires the dynamic linker to modify a word. MIPS has long since had a solution to this using the DT_MIPS_RLD_MAP entry to give a pointer to another section which is writable. It would be nice to have this functionality on other targets as well however. Right now many dynamic linkers do not support this layer of indirection so this can’t happen by default, it must be an option. Currently .dynamic gets mapped to a PT_GNU_RELRO section. PT_GNU_RELRO sections are not shared between instances of a executable however. It would be a nice optimization to share the .dynamic section of an application where possible. In particular we would like to have this option for Fuchsia. Other dynamic linkers, like musl, already have some support for this on non-MIPS targets as seen here: http://git.musl-libc.org/cgit/musl/tree/ldso/dynlink.c#n1629. In particular the DT_DEBUG_INDIRECT entry is recognized.
This change would refactor code that already exists for MIPS specific .dynamic sections and make them available to other targets. Mostly this would just require renaming some things and moving a few lines around. The biggest part of this change would be to add the flag. I propose it be a -z keyword called “rodynamic”.
If I understand correctly, your motivation to add an option to make .dynamic sections read-only is to share .dynamic sections in memory so that each process executing the same file doesn’t have to have a separate copy of a .dynamic section. But you would usually save just one page or two by doing that because .dynamic sections are usually very small. Even if they are large, they could be shared between processes by copy-on-write. What am I missing?
Thank you for confirming. But one page is too small, no? I think we are not that picky about memory footprint. For example, LLD and gold’s outputs are usually slightly different in size due to various reasons such as the differences how they order sections, common symbols, etc., but no one seems to care about that kind of negligible differences. Merging DT_MIPS_RLD_MAP into DT_DEBUG_INDIRECT seems like a good change, but adding a new command option to save one page at runtime seems overkill to me. (But I don’t know much about Fuchsia, so correct me if I’m wrong.)
The motivation is not only memory savings but also security: can-never-be-written is strictly better than RELRO in all cases. The biggest win is when .dynamic is the sole reason for having a writable segment at all. The distinction is fairly small for exploitability, but not negligible.
LLD already has several command-line options that are not supported by or are different from ld or gold, so this wouldn’t be the first one (and probably not the last). If LLD supported per-OS configuration, we would make this default for Fuchsia and wouldn’t need the new option, but since these don’t exists, it’s something we would handle through the Clang driver.
The motivation is not only memory savings but also security:
can-never-be-written is strictly better than RELRO in all cases. The
biggest win is when .dynamic is the sole reason for having a writable
segment at all. The distinction is fairly small for exploitability, but not
negligible.
I'm not even sure if it is strictly better to make .dynamic read-only by
default is better than relro. It seems almost the same to me. What are
attack scenario you can think of in which you can exploit only when
.dynamic sections are read-only from beginning?
Maybe Fuchsia OS does something special for executable that contain
read-only sections only?
LLD already has several command-line options that are not supported by or
are different from ld or gold, so this wouldn't be the first one (and
probably not the last). If LLD supported per-OS configuration, we would
make this default for Fuchsia and wouldn't need the new option, but since
these don't exists, it's something we would handle through the Clang driver.
Yes, we have bunch of options that are not supported by GNU linkers,
particularly for LTO. So adding new options is fine as long as doing it
makes sense. But the bar should be higher than implementing compatible
options.
Having no host OS-specific default options in the linker is more like a
feature than a lack of a feature. We rely on compilers to give appropriate
options to us. That makes the linker's behavior more predictable, and it
also makes it very easy to support cross-linking.
The motivation is not only memory savings but also security:
can-never-be-written is strictly better than RELRO in all cases. The
biggest win is when .dynamic is the sole reason for having a writable
segment at all. The distinction is fairly small for exploitability, but not
negligible.
I'm not even sure if it is strictly better to make .dynamic read-only by
default is better than relro. It seems almost the same to me. What are
attack scenario you can think of in which you can exploit only when
.dynamic sections are read-only from beginning?
Maybe Fuchsia OS does something special for executable that contain
read-only sections only?
I'd be interested in this as well, CFI relies on relro for its vtable
protection to work. Is the issue just the window of opportunity between
mapping the pages and protecting them?
LLD already has several command-line options that are not supported by or
are different from ld or gold, so this wouldn't be the first one (and
probably not the last). If LLD supported per-OS configuration, we would
make this default for Fuchsia and wouldn't need the new option, but since
these don't exists, it's something we would handle through the Clang driver.
Yes, we have bunch of options that are not supported by GNU linkers,
particularly for LTO. So adding new options is fine as long as doing it
makes sense. But the bar should be higher than implementing compatible
options.
Having no host OS-specific default options in the linker is more like a
feature than a lack of a feature. We rely on compilers to give appropriate
options to us. That makes the linker's behavior more predictable, and it
also makes it very easy to support cross-linking.
Note that you could avoid needing to add a new flag by defining a new
ELFOSABI for Fuchsia and making this behaviour conditional on it.
Jake Ehrlich via llvm-dev <llvm-dev@lists.llvm.org> writes:
Hi,
This is a proposal to add an option to lld that forces .dynamic sections to
be read-only.
The .dynamic section is almost read-only except for the DT_DEBUG entry
which requires the dynamic linker to modify a word. MIPS has long since had
a solution to this using the DT_MIPS_RLD_MAP entry to give a pointer to
another section which is writable. It would be nice to have this
functionality on other targets as well however. Right now many dynamic
linkers do not support this layer of indirection so this can't happen by
default, it must be an option. Currently .dynamic gets mapped to a
PT_GNU_RELRO section. PT_GNU_RELRO sections are not shared between
instances of a executable however. It would be a nice optimization to share
the .dynamic section of an application where possible. In particular we
would like to have this option for Fuchsia. Other dynamic linkers, like
musl, already have some support for this on non-MIPS targets as seen here: dynlink.c\ldso - musl - musl - an implementation of the standard library for Linux-based systems. In particular
the DT_DEBUG_INDIRECT entry is recognized.
Actually, that is just a nice name they use. It is only defined to non
zero in
The idea of having .dynamic be RW is that the .d_ptr fields can be
relocated. Not sure if any dynamic linker actually does it, but we
should check.
This change would refactor code that already exists for MIPS specific
.dynamic sections and make them available to other targets. Mostly this
would just require renaming some things and moving a few lines around. The
biggest part of this change would be to add the flag. I propose it be a -z
keyword called "rodynamic".
There’s one use case we have which is the vDSO. Our kernel has to be able to load this vDSO into every process, but we don’t want to include full ELF loader in our kernel, so we use the read-only layout for the vDSO. Combined with page-aligned segments which LLD uses by default, the logic for loading vDSO is trivial. We’re considering other use cases as well.
Correct, it eliminates the short window between mapping the page and protecting it. There’s also the memory saving; it’s only a single page per dynamic library which isn’t a lot, by if you multiply that by the number of processes times the number of shared libraries used by these it can become more significant.
One of the design principles we’re trying to follow is to make everything read-only, unless it has be writable. The only reason for .dynamic to be writable is DT_DEBUG which is something we never intend to support. FWIW in Fuchsia all we need is a read-only .dynamic without emitting DT_DEBUG altogether, but we wanted to make sure that this flag is also usable elsewhere hence implementing DT_DEBUG_INDIRECT which is already supported by musl as Jake pointed out.
I’d rather have a flag and have our driver set it. ELFOSABI requires driver support which means it becomes difficult to produce Fuchsia binaries with compilers that don’t have the right driver (e.g. we have some developers using a combination of GCC with LLD and this would be an issue for them). I’m also not sure how well is ELFOSABI supported with LTO?
Can we take a step back please? I think a much better approach would be
to make the whole DT_DEBUG/DT_MIPS_RLD_MAP optional, not moving from the
former hack to the latter hack. Systems that want to support this new
mechanism reserve an AUX vector similar to AT_BASE and let the dynamic
linker fill it in. No space required in the executable that way.
Still allow users to emit a read-only .dynamic section
Not emit DT_DEBUG in the case of a read-only .dynamic section
Do none of this DT_DEBUG_INDIRECT/DT_MIPS_RLD_MAP stuff. We can just allow people to use the AUX vector for this purpose if they need it.
So the changes to LLD would include
Adding a flag for read-only .dynamic
Changing the flags for .dynamic according to this flag
Not emitting DT_DEBUG if this flag is set.
I think we’d be fine with that option if everyone agrees to it. I already have a patch which does that in fact (it’s a very trivial change). Would everyone be ok with this change?
The motivation is not only memory savings but also security:
can-never-be-written is strictly better than RELRO in all cases. The
biggest win is when .dynamic is the sole reason for having a writable
segment at all. The distinction is fairly small for exploitability, but not
negligible.
I'm not even sure if it is strictly better to make .dynamic read-only by
default is better than relro. It seems almost the same to me. What are
attack scenario you can think of in which you can exploit only when
.dynamic sections are read-only from beginning?
Maybe Fuchsia OS does something special for executable that contain
read-only sections only?
There's one use case we have which is the vDSO. Our kernel has to be able
to load this vDSO into every process, but we don't want to include full ELF
loader in our kernel, so we use the read-only layout for the vDSO. Combined
with page-aligned segments which LLD uses by default, the logic for loading
vDSO is trivial. We're considering other use cases as well.
If you are writing an in-kernel ELF loader which doesn't support all
features, you can just ignore "w" bits of segments and map all segments as
read-only, can't you? It doesn't sound like you need a generic solution.
Petr Hosek via llvm-dev <llvm-dev@lists.llvm.org> writes:
One of the design principles we're trying to follow is to make everything
read-only, unless it has be writable. The only reason for .dynamic to be
writable is DT_DEBUG which is something we never intend to support. FWIW in
Fuchsia all we need is a read-only .dynamic without emitting DT_DEBUG
altogether, but we wanted to make sure that this flag is also usable
elsewhere hence implementing DT_DEBUG_INDIRECT which is already supported
by musl as Jake pointed out.
It might be easier to have an option for making .dynamic be read only
and the logic for emitting DT_DEBUG becomes ".dynamic is rw or mips".
BTW, dynamic linkers really never relocate the .d_ptr entries?
Jake Ehrlich via llvm-dev <llvm-dev@lists.llvm.org> writes:
Here's my understanding of Joerg's proposal.
1) Still allow users to emit a read-only .dynamic section
2) Not emit DT_DEBUG in the case of a read-only .dynamic section
3) Do none of this DT_DEBUG_INDIRECT/DT_MIPS_RLD_MAP stuff. We can just
allow people to use the AUX vector for this purpose if they need it.
So the changes to LLD would include
1) Adding a flag for read-only .dynamic
2) Changing the flags for .dynamic according to this flag
3) Not emitting DT_DEBUG if this flag is set.
I think we'd be fine with that option if everyone agrees to it. I already
have a patch which does that in fact (it's a very trivial change). Would
everyone be ok with this change?
One of the design principles we’re trying to follow is to make everything
read-only, unless it has be writable. The only reason for .dynamic to be
writable is DT_DEBUG which is something we never intend to support. FWIW in
Fuchsia all we need is a read-only .dynamic without emitting DT_DEBUG
altogether, but we wanted to make sure that this flag is also usable
elsewhere hence implementing DT_DEBUG_INDIRECT which is already supported
by musl as Jake pointed out.
It might be easier to have an option for making .dynamic be read only
and the logic for emitting DT_DEBUG becomes “.dynamic is rw or mips”.
BTW, dynamic linkers really never relocate the .d_ptr entries?
musl’s dynamic linker doesn’t and neither ours. Older versions of glibc’s dynamic linker do, but I’m not sure about the newer ones.
What are you guys using instead of DT_DEBUG?
We use other OS-specific runtime means to register that address.