Statepoint GC usage and calling conventions

I’m trying to write a GC that uses the statepoint-example GC strategy. I’ve read the documentation on GC’s and state points, but I still have some questions.

  1. The stack map format associates statepoints with instruction offsets. I figured I could find a statepoint by its association with the return instruction pointer on the call stack. Is the instruction offset of the statepoint just past the call instruction or is it just before? Or in other words, is the instruction offset exactly the return instruction pointer or is it one instruction before that?
  2. When using the tailcc calling convention, is there a standard location for the return instruction pointer? Is it the same as the C calling convention? Can I expect this location to remain stable across LLVM versions?
  3. The PlaceSafepoints pass adds GC polls to the IR. What is the purpose of a GC poll?
  4. The RewriteStatepointsForGC pass adds two calls to LLVM intrinsics the IR. Is there anything I need to do on my end wrt these intrinsics?
  5. The stack map format include both locations and live outs. Are all locations considered GC roots or only live outs? If all locations are GC roots, what is a live out?
  6. Locations in the stack map format include registers, direct, indirect, constant, and constant indices. I’m assuming registers indicate values stored in registers and indirect locations indicate a pointer into the stack. But direct locations are said to be “frame indices”, is this not just another pointer into the stack? Finally, for constants and constant indices, are these constants in the sense of integer constants, array constants, etc?
  7. To implement crawling the stack, it makes sense that I could use the stack size of functions in the stack map to calculate the stack frame’s address from the stack pointer. But when a function ultimately calls my GC’s alloc function, I have no way to determine the GC’s alloc function’s stack frame address because it’s obviously not included in the generated stack map.

I can answer some of these at least.

  1. The stack map format associates statepoints with instruction offsets. I figured I could find a statepoint by its association with the return instruction pointer on the call stack. Is the instruction offset of the statepoint just past the call instruction or is it just before? Or in other words, is the instruction offset exactly the return instruction pointer or is it one instruction before that?

I am not certain but if you use e.g. unw_get_reg to get the IP register of a caller’s frame, it will match the value in the call site table (at least on x64 and aarch64 anyway). I’m reasonably certain that this value corresponds to the call instruction but I am not certain.

  1. The PlaceSafepoints pass adds GC polls to the IR. What is the purpose of a GC poll?

I don’t use these (I have my own safepoint mechanism), AFAIK the idea is that this would be a point where you determine whether GC needs to run. For example you may have to pause all threads to run a GC, so you could do that check here.

  1. The RewriteStatepointsForGC pass adds two calls to LLVM intrinsics the IR. Is there anything I need to do on my end wrt these intrinsics?

Probably not; it should be replacing your calls with not just two calls but as many calls are needed to handle relocation of live values, and also replacing your call return values with a special call to extract that information from the statepoint call result. (Again I don’t use this one, I track live values myself and place my own safepoint calls.)

  1. The stack map format include both locations and live outs. Are all locations considered GC roots or only live outs? If all locations are GC roots, what is a live out?

The locations are where the live GC-able values are living. The live-outs set is not used by statepoint (see the statepoints doc for more information).

  1. Locations in the stack map format include registers, direct, indirect, constant, and constant indices. I’m assuming registers indicate values stored in registers and indirect locations indicate a pointer into the stack. But direct locations are said to be “frame indices”, is this not just another pointer into the stack? Finally, for constants and constant indices, are these constants in the sense of integer constants, array constants, etc?

IIUC direct locations represent GC-able objects which are actually on the stack, at the given offset to the stack pointer. Constants represent GC-able values that are actually constants (like null); I don’t recall these ever showing up in a real statepoint-produced stack map though.

Note also that there are two stack map entries per live value, where the first is the base pointer and the second is somehow offset from that (in practice they are often equal though). Also, at the start of the statepoint stack map there will be a bunch of constant values to give things like the statepoint version and some other information (see the aforementioned link for more info).

  1. To implement crawling the stack, it makes sense that I could use the stack size of functions in the stack map to calculate the stack frame’s address from the stack pointer. But when a function ultimately calls my GC’s alloc function, I have no way to determine the GC’s alloc function’s stack frame address because it’s obviously not included in the generated stack map.

Honestly I would stick with an existing stack-walking library if you can, when using LLVM. I referenced libunwind up above, which has stack walking capabilities and works pretty well. Otherwise you’ll have to know detailed information about the stack frame format (which varies by CPU and sometimes OS) in order to unwind it yourself.

IIUC direct locations represent GC-able objects which are actually on the stack, at the given offset to the stack pointer. Constants represent GC-able values that are actually constants (like null ); I don’t recall these ever showing up in a real statepoint-produced stack map though.

What about indirect locations?

From the LLVM docs:

  • The Locations within each record may either be of pointer size or a multiple of pointer size. In the later case, the record must be interpreted as describing a sequence of pointers and their corresponding base pointers. If the Location is of size N x sizeof(pointer), then there will be N records of one pointer each contained within the Location. Both Locations in a pair can be assumed to be of the same size.

Does this mean a location of size N × sizeof(pointer) references an array of N pointers? Or does it mean that there are N-1 following related location pairs? Or does it mean something else?

What about indirect locations?

Indirect locations, if I’m recalling correctly, indicate locations on the stack where a pointer to a GC-able object resides.

Make sure you’ve read both the StackMaps doc and the Statepoints doc. I had to refer back to both many times during my development process.

Does this mean a location of size N × sizeof(pointer) references an array of N pointers? Or does it mean that there are N-1 following related location pairs? Or does it mean something else?

I never ran into this situation, by my reading of this is that if you have an array of N GC-able pointers allocated on the stack, then the size divided by the single pointer size will give you the number of pointers to be aware of, while the address will point to the first one. But, it might be most helpful to actually produce such a stack map and examine it. You can read the generated stack map from an object file by hand using the llvm-readobj --stackmap command.