Block introspection and GNU runtime new ABI support

Hi everyone,

I've attached two diffs, which can be reviewed separately by probably don't apply cleanly in isolation because they modify the same files in a couple of places.

One adds support for the new (hopefully finalised this time...) GNU non-fragile ABI. This also removes the -fsender-aware-dispatch flag as the relevant features have been folded into the non-fragile ABI. I've updated the relevant test. I may have accidentally reverted some of the reindenting that was done since I started modifying this file and svn decided to get very confused about how merging was meant to work; I've tried to manually merge everything, but I may have missed a spot or two.

The other patch adds a -fblock-introspection flag. When this is specified, or when -fgnu-runtime is specified, an extra field is added to the block structure that contains an Objective-C type encoding of the block function. I've also updated one of the block tests to ensure that this field is only being emitted when the flag is provided.

All tests pass with both patches applied.

David

gnu_nonfragile.diff (34.1 KB)

block_introspection.diff (13.1 KB)

I'm not keen on changing the blocks abi... It is like changing struct layout on an OS that already has a given struct layout. I'll research this to see and see what we can do to accommodate the data.

Hi David,

Adding introspection information for blocks is something we'll probably want to do in the next abi as well, but we'll do it a different way. Are you sure you want to diverge on this?

-Chris

Hi Chris,

The other patch adds a -fblock-introspection flag. When this is specified, or when -fgnu-runtime is specified, an extra field is added to the block structure that contains an Objective-C type encoding of the block function. I've also updated one of the block tests to ensure that this field is only being emitted when the flag is provided.

Hi David,

Adding introspection information for blocks is something we'll probably want to do in the next abi as well, but we'll do it a different way. Are you sure you want to diverge on this?

It's a feature we want now, so if you'll let us know how you want to implement it then we can modify our runtime to simplify the compiler, but it is something that we want sooner rather than later (as I said when you originally announced blocks, ideally we'd also like full reflection in bound variables as we have with LanguageKit blocks, but that's probably more effort to implement than it's worth for the few cases where it's useful, and we're reasonably happy to just use Smalltalk for them).

We've only just released the updated runtime for testing, so nothing in the new ABI is set in stone yet - but we would like it to be relatively soon.

David

We will likely initially focus on providing introspection data on the captured variables rather than the parameters to the block. Both are of interest but with captured variable knowledge we can avoid generating copy helper functions for almost all cases.

The reserved field in the block descriptor is the slot I intend to use for captured variable knowledge.

Changing the ABI is very painful - we were unable to move the invoke function into the descriptor structure as we had originally intended for example, so I also advise caution.

Blaine

Hi everyone,

I've attached two diffs, which can be reviewed separately by probably don't apply cleanly in isolation because they modify the same files in a couple of places.

One adds support for the new (hopefully finalised this time...) GNU non-fragile ABI. This also removes the -fsender-aware-dispatch flag as the relevant features have been folded into the non-fragile ABI. I've updated the relevant test. I may have accidentally reverted some of the reindenting that was done since I started modifying this file and svn decided to get very confused about how merging was meant to work; I've tried to manually merge everything, but I may have missed a spot or two.

Checked in TOT: http://llvm.org/viewvc/llvm-project?view=rev&revision=81462

- fariborz

We will likely initially focus on providing introspection data on the captured variables rather than the parameters to the block.

This would also be of interest to us - we already have this with blocks generated from Smalltalk or JavaScript with LanguageKit and can access all of the variables in their scope using KVC.

Both are of interest but with captured variable knowledge we can avoid generating copy helper functions for almost all cases.

Yes, I can see this is would make life a lot easier.

The reserved field in the block descriptor is the slot I intend to use for captured variable knowledge.

We're not using that for anything at the moment (Remy's original code was using it to tell if a block was on the stack, but that's now been fixed to use the isa pointer, which I presume is the same solution you emply).

Changing the ABI is very painful - we were unable to move the invoke function into the descriptor structure as we had originally intended for example, so I also advise caution.

Yes, I completely agree, which is why I'd like to get it right before we get people deploying code with the Darwin ABI on other platforms. That said, we (which, in this context, means GNUstep) won't be making an ABI-stable release for a couple of months, so I don't mind breaking things again in the next few weeks if there are problems arising from this.

Fariborz seems to have committed the patches before any consensus was reached - I'd be happy if someone wants to revert it until everyone is happy with the proposed changes and I'll commit it later after any review comments have been addressed.

David

In the interests of future compatibility, may we use one of the bits in the flags field to indicate the presence of this type encoding (e.g. (2<<30))? Then we can easily switch block runtimes in existing code and just keep the accessor function for the type encoding (or compute it on-the-fly if that flag is not present but the information is there in some other way).

Longer term, we'd probably like to deprecate our blocks runtime and use the one from compiler-rt, but looking at that code it seems that it will take a bit of effort to make it compile on platforms other than Windows and Darwin, and to integrate with the GNU Objective-C runtime.

David

Ping?

It seems like non consensus was reached, but the code went in. Should
we pull it for now?

- Daniel

It hasn't gone in yet. I've modified my local copy to set flag 1<<30 on blocks that have this metadata attached to them, but I was waiting for confirmation that this wouldn't break anything for Apple before I committed it.

David

It hasn't gone in yet. I've modified my local copy to set flag 1<<30 on blocks that have this metadata attached to them, but I was waiting for confirmation that this wouldn't break anything for Apple before I committed it.

gcc uses BLOCK_HAS_DESCRIPTOR = (1 << 29) as the last used flag. However, runtime may have different ideas for
(1 << 30). So, answer should come from them.

- Fariborz

It hasn't gone in yet. I've modified my local copy to set flag 1<<30 on
blocks that have this metadata attached to them, but I was waiting for
confirmation that this wouldn't break anything for Apple before I committed
it.

I was referring to this:

Checked in TOT: http://llvm.org/viewvc/llvm-project?view=rev&revision=81462

and:

Fariborz seems to have committed the patches before any consensus was
reached - I'd be happy if someone wants to revert it until everyone is
happy with the proposed changes and I'll commit it later after any
review comments have been addressed.

David

- Daniel

That was my mistake; there were two patches, one about block introspection and one for the new GNU runtime ABI. The latter was committed, the former was not (I thought that, because they both touched the same files in a few places, committing one would be difficult, but apparently svn is more clever than I give it credit for).

David

I've had this sitting in my local copy for two months now and not heard any objections to using this flag. If there are any, please speak now before I commit...

David

-- Sent from my PDP-11

It hasn't gone in yet. I've modified my local copy to set flag
1<<30 on blocks that have this metadata attached to them, but I was
waiting for confirmation that this wouldn't break anything for Apple
before I committed it.

gcc uses BLOCK_HAS_DESCRIPTOR = (1 << 29) as the last used flag.
However, runtime may have different ideas for
(1 << 30). So, answer should come from them.

I've had this sitting in my local copy for two months now and not heard any objections to using this flag. If there are any, please speak now before I commit...

I sent a 2nd email to them. Waiting for the response. You can always check it in. We can remove it if they raise objections.

- fariborz

Fariborz, Daniel,

If the feature remains as initially described, that is, that when 1<<30 is marked in the flags field that an extra field shows up in the block itself containing the @encode of the block arguments for the gnu runtime, then I have no objection.

The direction is to put not only the block argument @encode description in place but also a concise block layout description such that the runtime can do all the of the copying work without helpers - except for the cases where C++ c/dtors need to be called. The layout will likely need to serve some new purposes or need to be flexible to serve new purposes in the future. The descriptor block is where this data will live since it is sharable and immutable.

I forget whether the GNU runtime @encode data is different than ours and so I don't know whether the GNU runtime will be able to use our data or at least wherever our slot does show up and how we mark it, but I think that is a not unreasonable goal to consider.

Blaine

Fariborz, Daniel,

If the feature remains as initially described, that is, that when 1<<30 is marked in the flags field that an extra field shows up in the block itself containing the @encode of the block arguments for the gnu runtime, then I have no objection.

Yup, that's exactly how it works at present.

The direction is to put not only the block argument @encode description in place but also a concise block layout description such that the runtime can do all the of the copying work without helpers - except for the cases where C++ c/dtors need to be called. The layout will likely need to serve some new purposes or need to be flexible to serve new purposes in the future. The descriptor block is where this data will live since it is sharable and immutable.

We are not making this field public outside of the runtime currently, we are just exposing it via runtime functions, so when you have something better we can easily swap over to using that.

This isn't set in stone for us at the moment (we haven't done a release of anything that depends on it yet and won't for a few months) so if there are any changes that would make this migration easier in future then I'd be interested in making them...

I forget whether the GNU runtime @encode data is different than ours and so I don't know whether the GNU runtime will be able to use our data or at least wherever our slot does show up and how we mark it, but I think that is a not unreasonable goal to consider.

It's exactly the same - we use the same code to generate it.

David

-- Sent from my PDP-11

-- Sent from my Difference Engine