It is vital to keep in perspective that Frame Pointer based stack traces can be imprecise. A previous study shows that about 5-7% stack traces (using FP method; on x86_64 and AArch64) can be incorrect with missing frames (Making sure you're not a bot!). The case of s390x is unique in that some of these issues are further magnified and FP method cannot be used ( Making sure you're not a bot! ).
This distinction is important, but it creates a fundamental problem: .eh_frame cannot be eliminated. It contains essential information for restoring callee-saved registers, LSDA, and personality information needed for debugging (e.g., reading local variables in a coredump) and C++ exception handling. Adopting SFrame would therefore require carrying both formats, resulting in a significant net size increase.
The median .eh_frame size across executables and shared libraries on a Linux system is already 5+% of total VM size. Doubling this overhead to ~10% by adding SFrame on top is simply not viable for most deployments.
In addition, Intel’s 11th Gen and AMD Zen 3 support hardware shadow stack.
A software-only stack walking approach (and remains unvetted for AArch64-see below) that doesn’t replace .eh_frame would quickly become obsolete.
Shadow stack can be enabled per process, providing flexibility to balance performance overhead / memory consumption with profiling needs, even for users who don’t prioritize the security hardening aspect.
While the imprecision of frame pointer-based unwinding is well understood, this doesn’t necessarily make SFrame the right solution. Alternative approaches exist, such as compact unwinding formats that can fall back to DWARF when needed, providing both size efficiency and improved precision. LLVM’s compact unwind format is one such example. Additionally, a straightforward enhancement to frame pointer unwinding—detecting prologue and epilogue patterns by disassembling instructions at the program counter—has not been explored.
On a related note, both the linux-perf-users post and https://developers.redhat.com/articles/2024/10/30/limitations-frame-pointer-unwinding#assembly_code_functions_in_libraries contain a small error: grep -v "[k]" should be grep -v "\[k\]" to properly escape the brackets.
Regarding the frame pointer prologue issue mentioned in the article, I collected my own data while running ninja check-llvm and observed only 1.5% of samples falling into the first 8 bytes of a function—notably lower than the 5-7% figure cited in the study.
I’ve raised concerns about SFrame viability for userspace stack walking: https://lore.kernel.org/linux-perf-users/87h5vg5tvj.fsf@oracle.com/T/#m0075aa5ef423df2f345ae682e9c8e815b06e6085
With the response at https://lore.kernel.org/linux-perf-users/87h5vg5tvj.fsf@oracle.com/T/#m95729a4f826fd045ca125d71ad1af36746c97393, we now know of objections from two of the ten most active contributors to tools/perf. I’ve reviewed other linux-perf-users discussions about SFrame and have not seen preference for it from other top contributors. I believe the size comparison data between SFrame and compact unwind formats could shift perspectives even for those who may have supported SFrame based on earlier, incomplete information. A major glibc maintainer has also expressed concerns about SFrame.
Additionally, I’ve chatted with mobile toolchain developers at the LLVM Dev Mtg, who emphasized that size concerns are especially critical for AArch64, which is heavily deployed on mobile phones. The lack of support from Arm ABI maintainers is also a concern—they are unlikely to want a format known not to work with mobile Linux to coexist with a future, more widely adopted compact format with callee-saved register, LSDA, and personality support.
Completely ignoring sframes for a moment:
If there is anything the proliferation of unwind and stack tracing formats should tell us, it is that every single one comes with a set of compromises, whether thoroughness vs size, loadable vs unloadable, and speed vs total unwinding and callind destructors or size, or whatever other tradeoff may be involved. It is highly unlikely that there is any format that without big compromises on multiple dimensions. The standard “useful for all unwinding and tracing use cases” is unmeetable.
So the fact that sframes may or may not be suitable for mobile or perf is really neither here nor there. No one should expect it to handle every use-case or every situation. Those who find its tradeoffs unpalatable don’t have to turn it on.
If anything, we should evolve it in such a way that it really fills its specific niche extremely well.
The question is simply: “Is the use-case that sframes address sufficiently important to a sufficient number of users to accept about 1,000 new lines of code in LLD?” Observe that the gnu folks have already made up their mind on this, and whatever anyone else’s reservations about the format may be, there are perhaps a dozen companies or so who have said they are interested.
Anyway, I don’t think we are seeing new technical arguments anymore, so this goes on the agenda for the next project council meeting.
This was discussed up thread and I have pointed out why this is not a viable solution for a lot of users. I will try to explain again.
Enabling shadow stack has non trivial negative performance impact. This is inherent in the technology itself – as it introduces memory accesses for every call and return and pollute the caches. The memory overhead can also be a problem for users who care about it – especially when the number of threads is high.
The shadow memory management can also be pain – how much do you need to config to avoid overflow?
It is a great technology, but it has been 10 years since its introduction and has not gained major traction yet for many reasons including the above.
The rest (majority) explained why the simplicity of Sframe is the key to its adoption in kernel. One of the objections you mentioned is about shadow stack – which is not an universal solution (see my reply above).
Responding to @MaskRay 's issue tracker comment here:
We clearly need to redesign the format from the ground up. The question isn’t whether SFrame fills a niche—it’s whether LLVM should upstream a format with fundamental architectural problems and significant objections when better alternatives exist or are emerging.
The completeness and acceptance of SFrame does not meet the standards LLVM has required, probably even among incubator projects.
The number of objections has been unheard of. Upstreaming at this early stage is inappropriate.
Working on a next-generation unwind format is a more promising direction; upstreaming SFrame would only hinder such efforts.
I accept the technical criticism of the sframe architecture. They are too large, limited, and redundant with eh_frame. Other platforms (Windows, Apple/MachO, ARM EHABI) have demonstrated that you can build general, compact unwind info formats. The only unanswered question I have is, how much slower do these formats make online profiling compared to FP unwinding (chasing a singly linked list up a hot stack is considered fast)?
However, I think you’re putting a very high cost on the burden of supporting a suboptimal file format, and I think that’s the question we should answer as a community. Is it reasonable for a lead maintainer to push back and just say “no” to a bad format that has users willing to maintain the feature? LLVM supports many limited, legacy object file formats, at some cost. But, as a project, we usually accept the cost of additional maintenance burden in exchange for increased contribution from the interested parties. Open source is stone soup.
I feel like there is a path here where LLD adds support for sframes, we gather implementation experience in the field, we feed those lessons into the next generation unwind info format, we deploy the next generation format, and deprecate this incarnation of sframes in two years. It sounds like @maskray wants that to happen, but downstream, or in an incubator, somewhere out of tree, to avoid maintenance burden. The Linux kernel epitomizes this development philosophy, but historically LLVM has favored upstream-first, incremental development, although perhaps this is just changing as the project matures.
I don’t think it makes sense for LLVM to accept patches to implement every format, feature, or extension under the sun. Some technical judgement and taste is required to decide what features are “heinous” and which are just a burden that must be borne. I hope we can come up with useful guidance for making these judgements going forward.
1 Like
Thanks Reid. I appreciate your acknowledgment of the technical issues, but I don’t think your response addresses the fundamental concerns.
You accept SFrame is “too large, limited, and redundant with eh_frame”—but then propose upstreaming it anyway.
This is not just about maintenance burden tolerance; it’s also about whether LLVM should commit to a fundamentally flawed architecture.
The “gather experience and deprecate in 2 years” path is wishful thinking.
Once SFrame is in LLVM and binutils, the incentive to redesign it disappears.
SFrame kernel developers could use this as arguments that “LLVM developers have endorsed this format.”
Why would anyone invest in a next-gen format when “SFrame works”? History shows deprecated formats persist indefinitely. We’ll be stuck with this suboptimal design, not iterating toward a better one.
In the optimistic case, Google server production adopts this, and Oracle Linux might adopt it as well. It’s difficult to see other systems adopting it with the current data. Once Google has adopted it, I can see it being very difficult to remove. llvm/lib/Target/Lanai is an example.
I believe that my suggestion of Post-processing alternative is a better alternative to “gather experience and deprecate in 2 years”
Post-processing alternative
A more cautious intermediate strategy could leverage existing Linux distribution post-processing tools, modifying them to append .sframe sections to executable and shared object files after linking completes. While this introduces more friction than native linker support and requires integration into package build systems, it offers several compelling advantages:
Allows .sframe format experimentation without imposing linker complexity
Provides time for the format to mature and prove its value before committing to linker integration
Enables testing across diverse userspace packages in real-world scenarios
Post-link tools can optimize and even overwrite sections in-place without linker constraints
For cases where optimization significantly shrinks the section, .sframe can be placed at the end of the file (similar to BOLT moving .rodata)
However, this approach faces practical challenges. Post-processing adds build complexity, particularly with features like build-ids and read-only file systems. The success of .gdb_index, where linker support (–gdb-index) proved more popular than post-link tools, suggests that native linker support eventually becomes necessary for widespread adoption.
You could even use non-SHF_ALLOC sections to make post-processing easy. (Tracers need to initiate I/O instead of relying on kernel’s page fault on-demand.)
SFrame developers can create such a standalone tool that parses .eh_frame and generates .sframe, then on the LLVM side no separate implementation is needed at all.
Google server production could integrate this in their linker wrapper.
This isn’t typical maintenance burden. There is significant cost in LLD, likely memory consumption and performance as well if you add metadata to every input section.
Then there is probably permanent 5+% VM size tax on every binary that adopts it.
Upstream-first assumes the design is incrementally fixable.
SFrame’s format deficiencies, object file format problems, and linker option design issues cannot be fixed with iteration.
This needs to be right from the start. Incremental development works for implementation details, not fundamental architecture.
I can see huge resistance for someone to fix the deficiencies as “it works”.
The level of opposition is unprecedented. We have objections from key perf maintainers, glibc maintainers, Arm ecosystem concerns, and mobile toolchain developers.
“Users willing to maintain” doesn’t constitute community acceptance when major stakeholders are pushing back.
Not many people are willing to participate in Discourse conversations.
Therefore, I left a comment on the issue tracker Add support for SFrame stack trace format · Issue #64449 · llvm/llvm-project · GitHub which better demonstrates external support.
The profiling performance question remains unanswered. You ask how much slower compact unwind formats are for profiling - but this is exactly the data we need before upstreaming, not after.
If compact formats perform adequately, SFrame’s entire premise evaporates.
The question isn’t about being overly conservative. It’s about whether we should upstream something we all agree is architecturally flawed, hoping it somehow improves later. That’s not incremental development—that’s technical debt.
I believe my linux-perf-users post, if read by perf stakeholders, will shift the perspective of those who are on the fence about SFrame based on earlier, incomplete information (I remember reading a statement like “.sframe is half of .eh_frame on x86-64” one or two years ago, which is obviously not true today).
I’m concerned this may become OKR-driven development. I’ve spoken with several Google folks at the LLVM Developers’ Meeting, and I hope they understand that thoroughly evaluating SFrame is already a substantial contribution that should be recognized. There’s no need to push through to completion with implementation matching current Binutils–that will likely need fundamental redesign anyway. The ecosystem would benefit far more from Google’s participation in designing a proper compact unwind format.
4 Likes
Accepting for the moment that the criticisms are valid, and that the opposition is not overstated:
Maskray’s request here is to create a new unwind format from scratch that is acceptable and useful for basically everyone and nearly every use case (If all of these objections apply, then the solution must handle them).
It must be small, simple, and fast. It must be loadable without consuming runtime resources. It must not be redundant with eh_frame. Possibly it should replace eh_frame altogether.
It must be created and added post-link, without special linker handling knowledge of the format, and it must not use any semantics unusual for elf (unlike, crel, lld-style symbol resolution, and relr). It must be easy for distros to adopt.
It must be nearly perfect in its first iteration because once its in place, the motivation to improve it goes away. No incremental solution should be upstreamed. All development must be done in some separate branch.
This envisioned project is a much bigger one than sframes, probably by an order of magnitude, maybe more.
That seems like an awfully tall order to me. I would guess a prototype is at least a year away at the very soonest. Probably much longer to something actually adoptable.
I guess it could get done. But I think it is extremely unlikely that it will.
So the decision here isn’t “SFames vs something better”, but “SFrames vs nothing at all.” So we will have use case we would like to address that can’t be met without the scope expanding beyond what is reasonably possible.
The only use cases that can be served are ones that literally the entire gnu/linux/android/libc world agree on.
That seems unfortunate to me from a project level.
1 Like
I don’t think this is a fair characterization.
SFrame seems targeted at the very particular use case of simple, fast, and comprehensive stack tracing (e.g. covering prologs/epilogs and PLT entries). It looks to be very well suited to this niche, but it’s also a fairly narrow niche. The SFrame binutils wiki itself mentions how stack traces are “fundamental in the implementation of run-time mechanisms like non-local jumps, exception handling and stack unwinding”, but SFrame can’t be used to implement any of those things. The amount of support needed in the toolchain should be weighed against the applicability of the feature; e.g. since you mentioned CREL and RELR, CREL can replace all other static relocations, and RELR covers ~90% of dynamic relocations in PIE/Bsymbolic-PIC cases, so they get you a lot of coverage. On the other hand, .sframe would be strictly additive, so it seems different.
If the format were more comprehensive, I imagine MaskRay might have a different inclination towards adding special linker support for it, and I don’t think designing such a format would be as tall of an order as you think, since we have a lot of prior art to draw on (there’s an active discussion ongoing on a different Discourse thread). It would undoubtedly end up more complex than SFrame for the stack unwinding case, but it could possibly end up being an acceptable amount of complexity for the additional coverage you gain. It’d take time to gain consensus, but the design would also be improved further during that process, and the end result would be much more broadly applicable.
That being said, I went through Comparing llvm:main...Sterling-Augustine:sframe · llvm/llvm-project · GitHub to see the scope of the changes. They’re a bit hard to read because some whole-file reformattings have snuck in there, but the LLD code seems generally self-contained and not touching any core components (e.g. increasing the size of any core data structures). I don’t know if that would change when e.g. the TODO around garbage collection is implemented, but there was discussion in https://groups.google.com/g/generic-abi/c/3ZMVJDF79g8 around using section groups to work better with linker GC, so hopefully not. I’ll ask some of the server-side folks at Meta to take a look and chime in on whether SFrame has value for them (to see if there’s broader applicability).
EDIT: Corrected “stack unwinding” to “stack tracing” above, thanks @ibhagat for pointing that out.
Correction: stack tracing, not stack unwinding
Sorry, I dont blame you. This statement was out of context, it was a misleading statement to begin with. I have removed it.
1 Like
This is overloaded with hyperbole, and the assertion that the “level of opposition is unprecedented” misrepresents the situation.
Apart from performance of stack tracing, complexity of stack tracer is also important. E.g., early tests using s390x support for SFrame have shown notable improvements ( Making sure you're not a bot! ) in both the sampling rate and the size of the recorded data when capturing call graphs with perf.
Its not just the Linux kernel and its facilities (perf, tracing), other applications also benefit from fast, low-overhead stack tracing when their usecase is of in-process stack tracing.
I’m just following along from afar and don’t really have a strong view. There’s one thing I wanted to comment on below, but as I’m writing this post I’ll take the opportunity to say how I’m glad we now have an appropriate escalation procedure for these kind of disagreements. I don’t think we’re likely to reach agreement going back and forth on this thread, so I feel it would be very positive for this to be discussed within the LLVM project council.
MaskRay, I really appreciate you further elaborating on your technical concerns with SFrame. It’s helped me to better understand your position and why it is more complex than “GCC supports this so we probably should too”, even if I’m not quite sure what conclusion I’d ultimately come to if I spent more time going through everything. But there’s something I wanted to comment on:
It’s possible I’m completely misinterpreting you here, but if you’re saying what I think you’re saying: please don’t make unkind speculations about people’s motivation for contributing something to LLVM. You have plenty of technical arguments and the experience and expertise to back them up, while this speculation doesn’t help your argument or help us to come to consensus as a community. Let’s just assume good intent from all parties.
1 Like
Thanks Alex. I appreciate the feedback and I understand the concern about tone.
To clarify: my comment about OKR-driven development wasn’t idle speculation. I’ve had direct conversations with Google colleagues at the LLVM Developers’ Meeting.
The impression I got was that they seemed committed to SFrame as a viable solution without fully considering existing alternatives, with a determination to upstream it.
However, they did agree that if better solutions emerge, they would consider them - which I genuinely appreciate and view as a constructive approach.
However, you’re right that focusing on the organizational pressure doesn’t advance the technical discussion.
On tone and productive discussion: I’ll be mindful of how I frame concerns. I’d also hope we can maintain balanced standards — dismissing technical questions as “nitpicking” or “rabbit holes” doesn’t foster the technical discussion we need either. Let’s all focus on the substance.
The core issue is timeline pressure. Whether driven by organizational goals, user commitments, or other factors, there seems to be urgency to upstream SFrame despite acknowledged fundamental issues.
My concern is that this urgency is overriding the normal technical review process that would typically prevent upstreaming something we agree is architecturally flawed.
The post-processing alternative I proposed provides a path to gather real-world experience without committing LLVM to supporting a format that may need complete redesign. This addresses the urgency while preserving technical judgment.
My takeaway from the linux-perf-users discussion is that significant concerns about SFrame have emerged.
https://lore.kernel.org/linux-perf-users/CAN30aBGX+CuPmPGRjzpRT69pP0QJc_zBAr69RqnMUZ-OXF=t1A@mail.gmail.com/
With these discussions bringing visibility to alternative solutions like compact unwind formats and their efficiency characteristics, perf maintainers are likely to raise the bar for accepting any stack walking mechanism.
All of this challenges the maturity of the current format and implementation, favoring incubation rather than direct contribution to the main branch.
2 Likes
I tried to avoid engaging with this kind of discussions, but you continue to spread your speculation and impression. Please stop.
The motivation is for better stack trace, profiling quality, and enabled optimizations. The alternatives are considered (and was explained many times in the thread). It is believed that the feature will benefit a lot though not all LLVM users.
Alignment with objectives does not equates urgency. There is a confusion about urgency vs project velocity. It is believed that velocity of project is important and the disagreement seems to be on the tradeoffs between velocity and perfection in design. Nobody is against the idea of reaching the optimal final state eventually.
Maybe the kernel folks (at Meta?) that worked on
Could provide some insight on this thread?
Some thoughts from my side:
As I already said above, the fact that a format has design defects is not in itself reason to reject support in LLVM. LLVM needs to support any formats/standards/etc that are required for wider ecosystem compatibility, regardless of our personal opinion on them.
However, at the time I wrote that comment, I thought that sframe is seeing much wider adoption than it actually is. I was under the impression that Ubuntu 25.10 is rolling out sframe in user space, but it turns out that this was actually just a toolchain misconfiguration.
I think MaskRay made a fairly compelling case for why sframe is not a great format for user space stack tracing. The size overhead is quite substantial, with sframe being around the same size as eh_frame/eh_frame_hdr, without subsuming it (that is, both are needed). I wouldn’t call it non-viable, but it’s a somewhat hard sell. Especially considering recent discussions and experiments with a compact unwind format, which suggest that we can actually do much better in this area. On the other hand, that alternative is still far away, and a vendor might want to take a sub-optimal solution now rather than a perfect solution later.
This does make sframe support in LLD appear less urgent to me. This would be critical for user space adoption, but kernel use cases can be satisfied in other ways. However, I’m also not very clear on who wants to adopt sframe where (and with how much buy-in) at this point in time. Is there a summary for that somewhere? (I’m not even really clear on what exactly the current state of support/adoption in places like the kernel/perf/etc is.)
Regarding waiting on sframe v3, I’m not sure how much that will help. Please do correct me if I’m wrong about this, but the impression I got is that sframe v3 is unlikely to address MaskRay’s primary concerns. I think from the points he raised, only support for large text segments is mentioned in the v3 doc. In particular, it’s my understanding that the concern about “ELF specification violations” is not going to be addressed, because the index building is considered fundamental to the purpose of the format.
Regarding maintenance concerns, I think these are somewhat overblown. Sframe support certainly adds some complexity to the linker, but especially with people volunteering to maintain this integration, I wouldn’t expect the burden to be excessive.
I believe that part of the motivation for rejecting sframe support in LLD is that this is essentially the “final point of intervention” for the sframe format. Once support sframe support lands, there is no way to exert pressure towards fixing issues with the format anymore. Though I’m not sure this pressure is particularly successful now either, and I would caution that it may be considered as hostile. These kinds of tactics may negatively impact future cooperation.
I think the larger problem here is that the sframe format was designed by GNU toolchain developers, without sufficient feedback from other important stakeholders (like MaskRay). I think we’d be in a much better place if these concerns had been brought up and discussed during the initial development of the format, rather than now. We have a lot of sunk costs now.
This is not to blame the GNU toolchain developers – I think this is a recurring problem we have, where either the GNU or the LLVM side comes up with some shiny new feature without talking to the other side, and then problems only become evident at a much later time. As far as I know, we currently don’t even have a good venue to have these cross-toolchain conversations (at least where they don’t intersect with some existing standards body). This got worse with LLVM’s migration to Discourse, as it’s now no longer possible to cross-CC mailing lists.
Overall, at this time, I’m neutral on this proposal. I think the case for sframe support is much less compelling than I originally thought, but at the same time, I also don’t think that adding sframe support to lld will be the end of the world.
2 Likes
Thanks Nick for sharing your thought on this and I strongly agree with your statement in the first paragraph and the point about maintenance concerns. It is reasonable to stay neutral on this proposal and keep this issue open. The landscape is still evolving (e.g. SFrame v3 adoption by kernel), and there will be more real world performance data (omitfp) coming up which will help us to make more informed decisions on the key trade-offs ahead.
1 Like
We are quite interested on the Gentoo side. I think Fedora is as well, but I can’t speak for them. I’ve had informal discussions where other distributions are interested as well. We don’t want to switch to using frame pointers, and we’re happy to experiment with it. The suggestions for changing .eh_frame are not going to quickly receive consensus, nor quickly be implemented.
I also think it’s on vendors to decide if the disk space trade-off is for them. From the measurements I’ve done for our environments, it looks like it would be acceptable indeed.
We have heard of alternatives on this for years and nothing has materialised except SFrames, which is why we are eager. I would have an open mind if an alternative format was implemented in future.
I have continued to see enthusiasm, especially on the kernel side, and at LPC this week, there’s a talk about it (not from the SFrame developers), as well as some other related discussions ongoing.
I don’t think it being in LLVM means that the format is considered perfect or has endorsement.
3 Likes
There is a persistent need for a stack walking format within the Linux kernel’s perf event subsystem, and the success of any such format hinges largely on kernel adoption. The question is: what will the different Linux kernel ports use?
ORC Unwinder is too restrictive—it’s limited to vmlinux and kernel modules (currently implemented only for x86 and LoongArch), has a larger format footprint than both .eh_frame and SFrame, and simply isn’t suitable for userspace.
SFrame evolved from ORC with the goal of supporting userspace. (There is ambition to eventually replace ORC, hence the problematic ld -r behavior for kernel modules, though some kernel folks prefer ORC’s simplicity and SFrame might not succeed there.)
It has been presented at multiple Linux conferences, yet the format has not achieved broad acceptance.
Notably, multiple top perf contributors have expressed reservations about its adoption.
Multiple folks told me they are uncimpressed by this format.
There is a remark that the Linux kernel will never integrate SFrame v2, but I have not seen agreement from Linux perf maintainers that SFrame v3 will be integrated either. You should really read v1/v2 as different stages of pre-releases, not an indicator of adoption.
(That said, less popular arch with fewer choices, such as s390x, might adopt SFrame.)
I want to add that the numerous presentations feel like an attempt to leverage the mere-exposure effect to get SFrame integrated into the perf event subsystem and Linux distributions (I have heard about semi-underground pushes for Linux distribution adoption, but I am unaware of any quantitative analysis supporting this effort.)
To my knowledge, SFrame was the only proposal currently in front of kernel developers. (Before I raised “Concerns about SFrame viability for userspace stack walking”) However, the SFrame design process lacked necessary input. In addition, the object file format aspects were missing critical considerations from regular GNU contributors familiar with the low-level details (binutils, gas, linker, .eh_frame). Nevertheless, it was merged into Binutils in November 2022 after a rubber stamp.
We now know that a superior alternative exists: compact unwind information with an asynchronous extension. This approach is intended to replace most of .eh_frame (unlike SFrame) and support exception handling. In contrast, SFrame is uninterested in exception handling. Supporting only stack walking—while competing with hardware stack walking features—puts it in an extremely narrow niche that is not viable for long-term sustainability.
I suspect much of the interest in SFrame was driven by misunderstandings of its capabilities. For example, I was also excited a few years ago by the claim that SFrame was about half the size of .eh_frame, which my subsequent measurements proved to be entirely untrue. As analyses like https://maskray.me/blog/2025-10-26-stack-walking-space-and-time-trade-offs become more widely known, I expect more people will become unimpressed.
I encourage the community to redirect their attention to compact unwind information, improve its design, and implement it. I hope this format becomes known to kernel developers and is evaluated fairly before we commit decisively to the SFrame approach.
For what it’s worth, I have successfully ported the Mach-O compact unwind format to ELF in an llvm-project branch, establishing a baseline for an asynchronous unwinding extension.
Unfortunately, I cannot dedicate full-time effort to this project—which is what would truly be needed to advance an alternative unwinding format. This contrasts with SFrame, which benefits from multiple full-time corporate headcounts, giving it a significant resource advantage.
Both formats require linker support, and maintaining both long-term is not sustainable. We need to choose the architecturally sound path once, rather than implement multiple competing formats.
FWIW, even for extensions with very limited adoption (e.g., used by just one or two companies), we typically face few, if any, major objections. However, the level of concern and widespread lack of enthusiasm for SFrame (of course there are also parties interested in it) is virtually unheard of.
2 Likes
The design space here is very large, with a lot of tradeoffs in many different directions. Compact unwind makes several tradeoffs that make it less suitable for the cases sframe is designed to handle. It might be fixable, but this would take some significant reworking.
Just one issue, for flavor: Compact unwind can’t represent a variety of abi-compliant stack-modification sequences, relying on compilers to generate specific sequences or, failing that, falling back to eh_frame to unwind in those scenarios. This means it is not standalone, and may not work for a variety of hand-written assembly code that sframes handle today. This also means its size and complexity is additive with .eh_frame, not a replacement for it.
This is an especially significant for the runtime unwinders themselves. To get the same stack-tracing support that sframe offers, the unwinder will need to be a full eh_frame unwinder as well as a compact-unwind-info unwinder.
That doesn’t make it a bad format at all–it’s a terrific format–just one that has a different set of tradeoffs than sframe.
One of Maskray’s early criticisms of sframe is that it is not concatenative and requires specific linker support. Compact unwind isn’t concatenative either, and requires specific linker support also.
I continue to maintain that the “level of concern” described above is dramatically overstated. I encourage everyone to go read the various comments both on the LLVM Discourse here, and those on lkml and observe who is objecting and how vociferously.
It isn’t clear to me at all that supporting both compact-unwind and sframes is unsustainable. That is entirely dependent on the amount of resources available, and who wants to spend the time. The fact that sframes have a significant amount of corporate support in its favor is a strong plus on that side. If this updated compact unwind info has enough people interested in maintaining it also, then it makes sense to support both.
5 Likes
Point of order.
I now feel compelled to call out the tone and framing of the arguments against SFrame by MaskRay. Describing the inclusion of SFrame in Binutils as a “rubber stamp” is unacceptable; it is extremely dismissive of the time and effort of the maintainers who reviewed and accepted the code, of community members who gave feedback in multiple conferences, of developers who have contributed to SFrame (which is more folks than just me). How can such language be acceptable in our communities ? I urge adherence to basic civility and maintaining some minimum standards of adherence to basic ethos of respect for contributions, collaborations and review processes.
6 Likes