LLJIT vs. thread-local storage

I am in the process of porting our ORC code to ORC v2 and LLJIT. Now that I have worked around a problem getting global constructors to be called, everything seems to work unless a module declares a static thread-local variable. In that case I get a “JIT session error” saying that the symbol __emutls_v.xyz was not found (substitute the mangled variable name for “xyz”).

Does that mean anything to anybody out there?

(I don’t know if it’s relevant, but we are using LLVM 8, and we are using Clang to compile C++ modules that are all put into a single JITDylib.)

Geoff

Hi Geoff,
Gathering from past, I remember that the ORCv2 doesn’t support thread local variable but not sure what is the current status now. What platform you are on?
CC’ed (Lang hames) he knows exactly what is the status.

Argh. Thanks for the info. We’re on Linux.

Oh, I think Linux don’t have support for TLS.

+Lang for visibility

This had also came up at llvm-devmtg briefly at the JIT roundtable. One of the collaborators on my project had started a patch years ago to implement some of it https://reviews.llvm.org/D8815, but then we went a different direction with TLS in our frontend and it became unnecessary.

And yet the same C++ code using thread-local variables works fine (or seems to) when compiled with Orc v1. Does the change to the Orc API really make thread-local storage more difficult?

Hi,
Orc v2 is different from the internal structure then Orc v1 not just in API level.

TLS support is not in ORC for a long time at least I’m aware of , Could you please confirm that ORC v1 actually compiles and run the code with Thread locals?

Hi Jameson,
Why thread local support is hard to support in JITs? Whether Julia supports Thread locals? If so, it would be very much helpful to know how.

Thanks

Yes, I confirm.

Can you able to share c++ code and your JIT setup?

Hmm, it’s a bit complicated. It will take some time, and I’m now on vacation for the next two weeks, I’m afraid…

Sure, no worries!

I don’t think it’s especially hard, but just not specifically unimplemented because nobody’s had a strong need for it. There’s probably some combinations of code models and machines that does happen to work (e.g. emutls+linux+large-code+large-data+no-PIC). Julia has some support for thread locals, but as a JIT in control of the language we currently try to generate better code than would otherwise happen (often bits of inline assembly that’s more similar to the initial-exec model than a dynamic library would normally be able to use)—but also with more limitations on how it may be used and requiring some additional support from the runtime.

TLS is normally handled somewhere between libc and dynamic linker. There
is currently no defined interface for requested another TLS segment or
just static TLS space.

Joerg

Thanks Jameson.

I think that’s both true and not. Yes, it’s not exposed through dyld to reuse its TLS routines (or, for that matter, its existing linker capabilities for in-memory objects). But I’ve also often seen that support to be a thin wrapper around pthread_key_create, which is defined in ISO/IEC 9945-1:1996 (``POSIX.1’'). Or you can build your own emulation for expandable TLS by putting a thread-id-counter in the original TLS, and using that to index your own internally-managed memory regions.

So, picking up this thread again after the holidays (burp)… Is there any chance that ORC v2 will support thread-local variables in the foreseeable future? Is there some way of configuring it today to ensure they are handled correctly? Or are we better off sticking with ORC v1?

(just adding Lang here for visibility)

Hi Geoff,

Thanks for your patience — I’ve been out on vacation and am still catching up on emails.

If this was working for you on ORCv1 then it could be as simple as ORCv2 not enabling TLS emulation by default (where I believe ORCv1 did) — I will investigate this tomorrow.

You said you were on an older LLVM version right? Whatever fix I make may have to be backported.

— Lang.