blocks and lambdas

I'm not interested in stepping on anyone's favorite technology. If you love Blocks and gnustep that is wonderful!

lambdas are part of the C++0x standard. Therefore, it is a mechanism all C++ compiler vendors will eventually support. My company does commercial desktop development and the "cross-platform" that matters to us is Macintosh gcc/clang and Visual-C++. It is highly unlikely that Microsoft will ever support Blocks (they don't even support C99)...particularly given they have already implemented lambda support in VC++. Therefore, it is in my interest to make sure Apple's tools offer first-class support for as many standard C++ technologies as possible so that my development burden is eased. I envision that lambdas will be a big part of my future and I wanted find out what direction clang is taking on lambda support.

<snip>
Unless you could persuade the C++ committee to break with tradition and specify an ABI, and specify that ABI to be compatible with the existing, deployed, blocks ABI, then this is not feasible.

And that is highly unlikely since the committee has already issued the Final Committee Draft and is technically not allowed to add new features. I guess you could wait another 5-10 years though. I believe proposals are always welcome.

http://herbsutter.com/2010/12/08/trip-report-november-2010-c-standards-meeting/

Ah yes, that's a good point. It would also be good to demote __block variables that aren't actually captured. Both of these would be easier if the variables listed the blocks that capture them.

John.

Lambdas and blocks are orthogonal technologies.

How are they orthogonal? I see a ton of overlap between the two…at least conceptually.

Blocks are a C extension designed to interoperate nicely with Objective-C and to work with C++. Lambdas are a (very different) C++ technology. Saying that you want a C library (libdispatch) to support a C++ technology (lambdas) doesn’t make much sense.

Why is the following structure incapable of describing a C++ lambda function?

struct Block_basic {
void isa;
int Block_flags; /
int32_t /
int Block_size; /
XXX should be packed into Block_flags */
void (*Block_invoke)(void *);
void (*Block_copy)(void *dst, void src); / iff BLOCK_HAS_COPY_DISPOSE */
void (*Block_dispose)(void ); / iff BLOCK_HAS_COPY_DISPOSE /
/
long params[0]; // where const imports, __block storage references, etc. get laid down */
};

If the clang C compiler can set up this structure for a Blocks block, I don’t see why the clang C++ compiler cannot do the analogous operation with a C++ lambda.

whether it is C or C++, it’s all just symbols and structures.

Given the different semantics between blocks and lambdas, implementing one in terms of the other does not really make sense. However, it would be fairly simple at the library level to implement a wrapper for one in terms of the other, so this isn’t a compiler issue, it’s an issue for your code.

Why do the semantics of the languages matter? They are conceptually the same thing. The only difference I see is the __blocks keyword/ block storage. My point earlier is don’t worry about block storage for lambdas. C++ programmers are expected to understand the lifetimes of their variables and not use them in ways inconsistent with their scope…also, if one wants to share variables with the stack and multiple blocks, he’ll declare the variable as a shared_ptr anyway. The block/lambda just needs to copy the shared_ptr by value.

Having blocks and labmdas share an implementation in clang would be problematic for a variety of reasons. For one thing, blocks already have a well-defined ABI, and lambdas will have a per-platform ABI. Unless you could persuade the C++ committee to break with tradition and specify an ABI, and specify that ABI to be compatible with the existing, deployed, blocks ABI, then this is not feasible.

The behavior can be specific to MacOSX where Apple controls the platform and ABI. I doubt there is extensive blocks use outside of the Apple universe. So, interoperability doesn’t matter outside the Apple universe. Apple doesn’t support newer versions of gcc that support lambdas and clang doesn’t do lambdas yet, so there should still be the freedom to pick a meaningful ABI for Apple platforms that makes the two technologies compatible. If Apple can change its ABIs to unify C++ and ObjC exceptions, then surely it can set up its ABI to make blocks and lambdas interoperable.

Bottom line…

I’m a 3rd party developer that has to share code between OSX and Windows…a description I’d say likely applies to the majority of mac developers. The clang team is building this compiler for people like me to use. My statement is…don’t make me, the customer, have to go through coding gymnastics to make my standard code work with Apple’s platforms. Don’t force me to use non-standard code structures for force me to have to write compatibility wrappers unless it is necessary. This stuff is extra work that makes my code more complex and less maintainable.

I’m not going to put on any pretense that I understand the technology necessary to make blocks and lambdas interoperable. I have a fair amount of knowledge, but this is not my area of expertise. I don’t daily work on compilers and libraries. I’m doing my best to communicate (as a general statement) to you guys who are compiler and library guys that we need to maximize interoperability with C++ standards and Apple-specific APIs so that people like me can deliver higher quality apps to market more quickly. I don’t think that is an unreasonable statement.

-James

Qua, 2011-02-09 às 15:48 -0800, James Gregurich escreveu:

>
> Lambdas and blocks are orthogonal technologies.

How are they orthogonal? I see a ton of overlap between the two....at
least conceptually.

> Blocks are a C extension designed to interoperate nicely with
> Objective-C and to work with C++. Lambdas are a (very different) C
> ++ technology. Saying that you want a C library (libdispatch) to
> support a C++ technology (lambdas) doesn't make much sense.

Why is the following structure incapable of describing a C++ lambda
function?

struct Block_basic {
    void *isa;
    int Block_flags; /* int32_t */
    int Block_size; /* XXX should be packed into Block_flags */
    void (*Block_invoke)(void *);
    void (*Block_copy)(void *dst, void *src); /* iff
BLOCK_HAS_COPY_DISPOSE */
    void (*Block_dispose)(void *); /* iff
BLOCK_HAS_COPY_DISPOSE */
    /* long params[0]; // where const imports, __block storage
references, etc. get laid down */
};

It can't, a lambda can be used as a function pointer!

Lambdas are syntactic sugar for function objects, which is fundamentally
different from what blocks are. What you are asking is lot of baggage
for nothing, from a pure C++ perspective, just to be compatible with
blocks.

A function object is a class with a set of data members and a callback function called ‘operator()()’.
A ‘block’ is a callback function and a set of captured data items.

How is that different?

consider…

#ifdef BLOCKS
void
dispatch_sync(dispatch_queue_t dq, void (^work)(void))
{
struct Block_basic *bb = (void *)work;
dispatch_sync_f(dq, work, (dispatch_function_t)bb->Block_invoke);
}
#endif

DISPATCH_NOINLINE
void
dispatch_sync_f(dispatch_queue_t dq, void *ctxt, dispatch_function_t func)
{

func(ctxt);

What do I see in this code and Block_basic? I see a callback function (func = Block_invoke), a ‘this’ pointer (ctxt is a Block_basic instance), a destructor (Block_dispose), and a cloning operation (Block_copy). I even see a ‘isa’ pointer (analogous to vtable pointer). What is does a single-inheritance C++ class have that this mechanism doesn’t have?

If all you do is set the system up so that if a lambda-to-block conversion is requested, the Block_basic structure is set up and mapped to the lambda, that would be enough. If you did that, then lambdas could be passed to libdispatch and the various cocoa/foundation functions that accept blocks as arguments. The developer writes his lambda and passes it to the apple function that accepts blocks…oblivious to the fact that a conversion has taken place. In the case where the lambda is entirely known at compile time and it is known that is is only converted to a block, then you can optimize the situation by just making it into a block at compile time.

I really don’t understand what the objection to the concept is. The technical objections I’ve seen don’t seem substantial to me when I look at the details. Do you think easy interoperability between standard technologies and platform-specific technologies is not a desirable goal?

Qui, 2011-02-10 às 11:54 -0800, James Gregurich escreveu:

> >
> > Why is the following structure incapable of describing a C++
> > lambda
> > function?
> >
> >
> > struct Block_basic {
> > void *isa;
> > int Block_flags; /* int32_t */
> > int Block_size; /* XXX should be packed into Block_flags */
> > void (*Block_invoke)(void *);
> > void (*Block_copy)(void *dst, void *src); /* iff
> > BLOCK_HAS_COPY_DISPOSE */
> > void (*Block_dispose)(void *); /* iff
> > BLOCK_HAS_COPY_DISPOSE */
> > /* long params[0]; // where const imports, __block storage
> > references, etc. get laid down */
> > };
> >
>
> It can't, a lambda can be used as a function pointer!
>
> Lambdas are syntactic sugar for function objects, which is
> fundamentally
> different from what blocks are. What you are asking is lot of
> baggage
> for nothing, from a pure C++ perspective, just to be compatible with
> blocks.
>

A function object is a class with a set of data members and a callback
function called 'operator()()'.
A 'block' is a callback function and a set of captured data items.

How is that different?

The problem is the "set of captured items". If a lambda does not capture
any items it can be used as a function pointer, I don't think you can do
that with a block since it carries baggage with it even if you don't
capture items. It would also violate one of the fundamental design
philosophies behind C++: "you don't pay for what you don't use".

fair enough. but, you can still provide for automatic conversion in cases where one wants to feed a lambda to libdispatch or some some cocoa/foundation class. As I said earlier in this thread...if you can make that happen, I'll be happy.

The goal that I'm lobbying for is to allow 3rd party mac developers to keep as much of their code in standard syntax and built upon standard libs as possible. Not only does it make the lives of existing mac developers easier, but it lowers the barrier to developers from other platforms to come to the mac.

Qui, 2011-02-10 às 12:44 -0800, James Gregurich escreveu:

yes. I am used to have to roll my own glue to make standard code with with Apple APIs. However, it would be nice if Apple's dev tools and APIs were structured to just work with the standards and not put the end user through hoops to coerce cooperation.

A good example of such hoop-jumping was with the integration of ObjC and C++ exceptions. If the unification had been done at macOSX 10.0, then I would have been spared writing macros to try and mangle the two systems cleanly on both the the 32 bit macosx case and the opposing case. I'd like to avoid that kind of crap. It makes my code harder to maintain and debug.

I'm coming to the engineers working on a brand new C++ compiler before they have implemented much of this stuff and I'm saying....think this problem out NOW and make it nice BEFORE you have a legacy code base to worry about....integrate it and make it work well so that we can have a nice, standards-based system.

I really don't understand why that is a statement that inspires objection. It seems to me that clean, tight, well-designed integration between Apple technology and standards-based technology should be a goal. I am encouraged by the fact that one of the Apple engineers is thinking about this concept and would like to try to make it work. I will be filing a feature request on ADC to officially log my desire for clean, automatic integration of lambdas with with Blocks-based APIs.

In general, requests for features that involve Apple APIs and technologies should go through Apple’s support channels (ADC). This list concerns the open-source Clang project, where Apple is but one of many concerned institutions.

With respect to lambdas and blocks, I too would like to see some level of interoperability. As John mentioned, an implicit conversion from a lambda object to a block pointer seems completely reasonable, and would allow one to seamlessly [*] use C++0x lambdas as arguments to APIs that take block pointers. Blocks can already be used with many APIs that accept C++0x lambdas, since a block behaves like a C++ function object so long as one handles the block copy/release externally. I doubt we’ll see any low-level compatibility between the two features, e.g., C++0x lambda objects aren’t likely to ever have the same layout as blocks, since C++ programmers won’t want that memory cost for simple lambdas.

That said, nobody has actually looked into the implementation of lambdas in Clang, and I haven’t seen any indication that it is coming soon. There’s not much more we can do until that happens.

  • Doug

[*] Subject to sensible capture semantics, which is complicated by the fact that blocks and C++0x lambdas capture referenced variables in very different ways. This is something that needs to be carefully designed offline.

thanks. I've already submitted a feature request through ADC. I'm confident that you guys can make it happen.

-James