This is just pedantry about terminology. Everyone knows perfectly well that null doesn’t point to an object in the sense that the standard uses that word.
However, its other properties (any two null pointers comparing equal, no other pointer comparing equal to a null pointer) are the same as if it were a pointer to a singleton object.
What is your definition of “object” then, if not what C says it is?
I hoped it would be clear from my use of the word “conceptual” that I was not using the word to mean what C says it is. Sorry for confusing the issue.
In my analogy, an object is the thing referenced by a pointer value, or the pointee if you prefer.
_Optional means that a pointee may not exist, const means that a pointee may be read-only and volatile means a pointee’s value may change unexpectedly. None of those properties preclude the pointer’s value from changing or mean that it must point to an object of that type (optional, const or volatile).
The fact that an _Optional object may not be able to be copied, initialised, accessed byte-by-byte, etc. doesn’t preclude it from existing as part of the language’s syntax any more than a pointer to void.
In my original idea, I didn’t bother forbidding declarations such as
_Optional int x;
because “may not exist” doesn’t mean “cannot exist”, and fewer rules are preferable. I changed that based on feedback from colleagues and on the forums.
I originally intended _Optional to be usable in any part of a
declaration and only banned its use at top level based on
feedback from others.
One interesting thing pointed out by a Redditor is that banning usage
of _Optional at top level could be justified better if so-qualified
values could not exist as the result of dereferencing a pointer, yet I
proposed no change to the semantics of *, -> and [] to remove any
_Optional qualifier from the result type.
I’d be just as happy for pointer dereference operations to implicitly
remove the _Optional qualifier from their result. I already have code to
implement that behaviour for Clang kicking around on a local branch
somewhere, and I wrote all the wording for the alternative proposal
but ultimately didn’t use it. 99% of programs using _Optional would
work without modification with either implementation.
The reasons that I specified that & should remove the _Optional
qualifier instead of *, [] and -> are given in my paper, but they aren’t
incontestable.
I’m not aware of any such requirement. Did you find that documented somewhere? It isn’t clear to me that _Nullable semantically differs from your proposed _Optional.
That appears to have come from one person in a code review that appears to have been accepted (and which includes changes to switch from the GNU attribute syntax to the Clang qualifiers). Do you have evidence that these qualifiers have not proved useful? I haven’t verified, but my understanding is that Apple has rather widely deployed them in their framework libraries and diagnoses them in their Xcode IDE.
I think it is interesting that the linked code review is for a change that deploys both _Nullable (10 occurrences) and _Nonnull (130+ occurrences) (_Null_unspecified was not deployed). This is just one data point of course, but it has _Nonnull deployed more than 10 times as often as _Nullable.
Unless I missed it, I don’t see any technical criticism of Clang’s existing qualifiers.
I’m afraid I don’t see a conflict here. Declarations may have the restrict qualifier present; they just aren’t required to.
I don’t find the example code presented compelling. Were you to show a before/after example comparing _Optional with _Nullable, the code would look very much the same; the proposal doesn’t offer anything that we can’t write today.
I find the proposed semantics for &* confusing. I think it would be challenging to explain to someone why, given a variable p declared as _Optional const int *volatile p, the expression &*p yields a type of const int *. The mechanics you desire for _Optional are already in the language.
My take away from this proposal is that we should work towards standardizing _Nullable and _Nonnull assuming there is sufficient motivation to standardize anything at all.
Thank you for putting together this RFC! For others, something along these same lines was previously proposed by ISO C3X proposal: nonnull qualifier which did not end with strong consensus but did have significant interest from folks in the community.
I have specific comments below, but a summary of my current position is: despite the importance of this topic, I don’t think it should be added to Clang at this time. I disagree with the syntactic choice made to put the qualifier on the pointee rather than the pointer. This design space is littered with attempts to solve the same problem and it’s not clear that this approach is going to be an improvement. The proposal is largely experimental in terms of design and needs a stronger indication that a standards body really believes in this design (in terms of syntactic choices, at the very least) and data demonstrating how this feature catches bugs that cannot be caught by other features in the same space (not just due to missing diagnostics that are possible to implement). The proposal also needs to work for all pointer types without requiring the user to jump through hoops (like using typedefs).
The existing features in this space that Clang supports are, at least:
The proposal touches on why the author believes some of these are deficient, but it doesn’t really talk about which ones are repairable in terms of diagnostic behavior or how the new functionality will interact with existing functionality. More examples comparing and contrasting the existing features with the proposed feature could perhaps help make this more clear.
In terms of syntactic choices, as mentioned above, I disagree that the qualifier belongs on the pointee rather than the pointer. Losing the qualifier on lvalue conversion does not seem to be an issue, as after lvalue conversion the loaded value can never be changed. So once you’ve done an lvalue conversion to read the pointer value itself, optionality of the pointee is resolved (so dropping the qualifier should not lose any diagnostic fidelity).
There’s also some details missing about how the feature plays with various languages. In Objective-C, object pointers are a bit special; should this qualifier be usable for them? In C++, if a member function is marked as being _Optional, what are the semantics? Separately, do you expect this qualifier to be mangled as part of a function signature? Can users in C++ overload functions (or in C with __attribute__((overloadable))) or create template specialization based on this qualifier? In C, how does type compatibility work? e.g., are these valid redeclarations? void func(int *i); void func(_Optional int *i) {} (Note, if these are not valid redeclarations, that’s another difference in behavior from other qualifiers in that position and is likely to cause confusion.) What is the ABI of passing an optional pointer and does it differ from the ABI of passing a regular pointer? What happens if one TU declares an extern pointer variable and another TU defines it as being an optional pointer (or vice versa)?
Despite my current feeling of “we should not add this at this time”, I think it’s useful to continue the discussion and iterate on the design to see if we can come to something that we think should be added at some point in the future.
This isn’t fully accurate. As you note below, static array extents are the way in which you signal that a pointer value cannot be null. However, adoption of static array extents in industry has been slow, common tooling often misses helpful diagnostics, the syntax is currently limited to only function interfaces, the syntax is awkward for void pointers, etc. But the syntax and desired semantics do exist in C already today.
The concerns around applying to parameters is a non-issue now that C has [[]] style syntax (and also given that Clang supports __attribute__ syntax on parameters directly). Also, intrusive and verbose are personal style concerns rather than technical issues with the functionality, so “I want this to be a keyword because I don’t like the way attributes look” is not very compelling, especially given how often attributes wind up hidden behind macros.
I think all pointer types already have the property of being able to represent a null pointer value. From that perspective, a type qualifier is unnecessary, that’s just the way the language already works today. However, being able to signal (to readers, to an analyzer, to the optimizer, etc) that a pointer value is expected to never be null is useful. The converse, a type signaling that a pointer is only ever null, is already handled by the nullptr_t type in C2x. But any annotation that says “this pointer might be null” seems like a non-starter because that is how pointers already work, so incremental adoption would be very unlikely.
Hmm, this has nothing to do with pointers though. This is how lvalue conversions work, and it works that way on everything, not just pointers. This is observable via _Generic already today (try getting it to associate with a qualified type, it won’t happen unless the qualifier is on another level of the declarator).
I don’t agree with that assessment. They’re qualifying the pointer to give information about what values the pointer may have, not what objects they point to. So, to me, they’re qualifying exactly what I would expect. And lvalue conversion gives you exactly the properties I’d expect – after obtaining the value of the pointer itself, you no longer need any marking to say whether it’s nonnull or not because you already have the value and it’s too late to ask that question. Basically, lvalue conversion is the point at which you would test whether the pointer could be null or not. It can never change state after lvalue conversion.
I’m not sold on the name given the current proposal. To me, _Optional has very little to do with pointer types and everything to do with values. e.g., I would not want to close the door on being able to write:
_Optional int get_value(struct something *ptr) {
if (ptr)
return ptr->field;
return _None;
}
int main() {
_Optional int val = get_value(nullptr);
if (val)
return *val; // Steals C++ syntax, but the * could be replaced by another syntactic marker
return EXIT_FAILURE;
}
this is more in line with Python’s optional functionality, as well as the same idea from C++ (though they solved it with a library feature rather than a language feature).
That said, I think:
int * _Optional get_ptr_value(struct something *ptr) {
if (ptr && ptr->field)
return ptr->field;
return _None;
}
is along the same lines of what you’re proposing, except that the optionality is lost on lvalue conversion. But again, once you have an rvalue, the state of the original object cannot change in a way that effects the loaded rvalue and so losing the _Optional qualifier is not harmful.
To be honest, if you designed _Optional to be less about pointers and more about values in general, I think the feature becomes much more compelling.
This is novel in C; no other qualifier works this way. So I’m not convinced that this is going to be less incompatible, confusing, and error-prone than other solutions. Also, const doesn’t always indicate anything about an object’s address (it can in theory for globals, but doesn’t for things like function parameters).
This continues to be an unresolved issue showing the irregularity of the syntax choice of the proposal to qualify something other than the pointer. The fact that you need to jump through hoops for function pointers, specifically, is a serious concern – these are pointers I would expect people would very much want to annotate if they’re plausibly going to be null pointers.
FWIW, the reason for the limitation against qualified function types in C is because functions are not data, but the function pointer is. That’s why you can qualify the pointer but not the function type itself.
Ok so the proposal has a lot of details but the core idea is relatively simple, I’ll split it in two parts:
An attribute to annotate pointers that need to always be checked for null before every use.
A much stronger programming model that assumes all unannotated pointers to never be null.
I’m assuming that this proposal doesn’t target C++, because C++ has check-required-pointers at home (check-required-pointers at home: std::optional<std::reference_wrapper<T>>).
I think (1.) is a reasonable proposition. Pointers that deserve such annotation definitely exist. For example, standard function gets() that may unpredictably fail with input/output error would be a great candidate for this. It’s going to be a much harder sell to put these annotations on pointers that may be null predictably (eg., “the result is null when the input is null, otherwise it’s always non-null”), as it’ll start flagging existing code that relies on that contract. But even in this case, I can totally see some projects committing to cleaning up their code according to that programming model.
Solution (1.) can be useful without (2.), as a standalone feature.
I think (2.) is insanely expensive, and the amount of people who can realistically write code like that, is miniscule. It is roughly equivalent to emitting a warning every time an unannotated pointer is assigned null (which is forbidden) or checked for null (given that it can’t be null, so the check doesn’t make sense). That’s an extremely aggressive approach that would flag enormous amounts of perfectly correct code. The adoption cost will in most cases outweigh the benefits.
Aside from annotation burden, code refactoring associated with (2.) may introduce more bugs than it fixes. For example, if a non-null (i.e. unannotated) pointer is initialized with non-trivial control flow:
int *x = NULL;
if (condition()) {
// expensive computations
x = ...;
} else {
// different expensive computations
x = ...;
}
use_nonnull(x);
Then the initial null-initialization can no longer be performed, so a programmer who tries to minimize code changes caused by keeping the pointer non-null will be tempted to simply drop the initializer:
int *x;
// same code below
Then if one of the branches forgets to initialize the pointer, you’ll end up with a garbage pointer that’s much more terrifying than a null pointer.
Separately, while it may be possible to annotate all your project code, I don’t see how you can easily extend this contract on third-party code that you’re using. Sometimes you really need to pass a null pointer into a third-party library function (even into a C standard library function) and you may be in a situation when adding such annotation into that library may be problematic.
So I strongly suspect that this approach will need massive quality of life improvements which will ultimately make it much more in line with existing solutions, which were the root cause of your frustration in the first place. There’s good reasons why existing solutions don’t simply do what you propose.
So that’s my initial opinion. Let me make a few other less important points here and there.
Well, if it was a property of the pointed-to object, then it would also make sense to write the same with zero stars:
const int i; // i is an int that _is_ stored in read-only memory
volatile int j; // j is an int that _is_ stored in shared memory
_Optional int k; // k is an int for which no storage is allocated???
Given that this doesn’t make sense for your _Optional qualifier, it sounds to me as if you just introduced a different way to spell a pointer qualifier (“let’s put it before the *, not after the *”) without any substantial differences to semantics.
You probably already noticed that but for the other readers, worth bringing up that this is the feature available in clang itself under the clang --analyze flag (aka Clang Static Analyzer), which gcc -fanalyze tries to mimic, and clang-tidy simply imports as-is. This check is also ancient in our case, it’s much older than the nullability annotations _Nonnull and _Nullable (which suggests that you have a typo in your example, it has to still be __attribute__((nonnull)), godbolt).
Well, no, it’s a property of a given group of pointers to the object. Just because an object has many pointers point to it, i.e. generally participates in aliasing, doesn’t mean you can’t pass it as an argument to
memcpy(void *restrict dst, const void *restrict src, size_t n);
It only becomes a problem when both pointers passed to memcpy() point to the same object. So, just these specific pointers, regardless of every other pointer in the program.
First of all, I don’t think this specific problem even needs an in-code solution at all. Instead, the analysis that causes the warning to appear near strcmp can be made smart enough to recognize that s1 and s2 are never null at this point. You need such smarts in the analysis anyway, to cover another very important case:
int foo(_Optional const char *s1, _Optional const char *s2)
{
if (...) {
return 0;
}
assert(s1);
assert(s2);
return strcmp(s1, s2);
}
This is a relatively easy flow-sensitive analysis that you can conduct over Clang CFG, or ClangIR whenever that becomes available. Such analysis would provide very good quality of life. Of course you can go for a purely syntactic solution instead, which will have cleaner rules but be worse in terms of quality of life.
Now, obviously, sometimes you really need a “force-unwrap” operator to indicate that you’re sure the pointer can’t be null here. In this case “easy to type” isn’t necessarily valuable; say, Rust chose the syntax .unwrap() which is designed to catch the eye, be easy to notice and audit. On the other hand, Swift uses ! which is, yes, easier to type. So what you can do is a compiler builtin:
_Optional int *x = ...;
int *y = __builtin_unwrap_optional(x);
Then if users want something fancier, they can have a macro, which can even include a debug-mode check:
#define unwrap(x) \
({ \
__typeof(x) __tmp = x; \
assert(__tmp); \
__builtin_unwrap_optional(__tmp); \
})
_Optional int *x = ...;
int *y = unwrap(x);
The assert is very useful because it’ll diagnose the problem at the moment of unwrap, not at the moment of dereference, which may happen much later.
So like I said in the other thread, I think this attribute doesn’t need to be checked by the static analyzer. The contract behind your attribute can be much simpler, fully resolved with either purely syntactic analysis or with very basic flow-sensitive analysis.
The static analyzer can take advantage of it. You can introduce a warning about any unchecked dereference of the _Optional pointer. The easiest way to introduce such warning is to perform a state split every time the pointer is encountered: in one state the pointer is null, in the other state it’s non-null. Then the null case simply becomes a path that the analyzer has to explore.
However, again, if you simply implement a simpler syntactic or flow-sensitive warning in the compiler, the analyzer work becomes redundant, because the problem is entirely defined away by the compiler warning.
Maybe there’s still room in the analyzer to warn about invalid force-unwraps, but most of such warnings would be about potential execution paths that the developer has just explicitly said aren’t there, aka false positives. Same reason we wouldn’t warn about asserts that fail on certain execution paths: the developer believes this execution path doesn’t exist, and even if it does, there’s already dynamic analysis (i.e. assert()) for this exact problem.
I would like to see a new type qualifier to indicate nullability.
At the moment I put attribute((nonnull)); on the end of functions (I don’t number usually, so that I start by default checking all are valid.
I compile using GCC -fsanitizer and null,returns-nonnull-attribute,address
-Wnonnull
-Wconversion-null
-Wno-nonnull-compare
Ok so the proposal has a lot of details but the core idea is relatively simple
Thank goodness someone noticed! I don’t think I could possibly have made a simpler proposal.
I’ll split it in two parts:
An attribute to annotate pointers that need to always be checked for null before every use.
The annotation spreads virally across all pointer assignments, including through function calls.
Yes, exactly like const.
The programmers are only allowed to use them inside an if-statement with a null check, or possibly after an assert.
Assertions mean different things in different contexts.
Conventionally, code to check for assertion failure isn’t included in release build configurations, so I’d expect assertions to be ignored for static analysis purposes (since they don’t protect anything). That could be done using macros.
Conversely, if assertions aren’t omitted (for example in test code) then I’d expect the static analyzer to recognize the longjmp, abort or equivalent that they wrap and treat following pointer dereferences as properly guarded.
A much stronger programming model that assumes all unannotated pointers to never be null.
People may assume that unannotated pointers are never null; the compiler must not.
I think a lot of people already do assume that pointers are not null (based on comments like “I love null pointers - they are guaranteed to generate a SEGFAULT!”)
I make the same assumption in most user space code except that I’ve been Stockholm-syndromed into adding
XXXX_ASSERT_POINTER(x);
XXXX_ASSERT_POINTER(y);
XXXX_ASSERT_POINTER(z);
on entry to every function with pointer arguments. I assume that most normal people don’t do that – either because they don’t know or care what undefined behaviour is, or because life is too short.
It’s going to be a much harder sell to put these annotations on pointers that may be null predictably (eg., “the result is null when the input is null, otherwise it’s always non-null”), as it’ll start flagging existing code that relies on that contract. But even in this case, I can totally see some projects committing to cleaning up their code according to that programming model.
Yes, this is the value proposition: the human writes more defensive code, and in return the compiler helps the human not to make stupid mistakes. Still, it’s entirely up to the individual to decide which functions use the new qualifier.
Solution (1.) can be useful without (2.), as a standalone feature.
I agree. Thank you.
I think (2.) is insanely expensive, and the amount of people who can realistically write code like that, is miniscule. It is roughly equivalent to emitting a warning every time an unannotated pointer is assigned null (which is forbidden) or checked for null (given that it can’t be null, so the check doesn’t make sense).
That’s why it wasn’t part of my proposition, although a surprising number of people have been telling me that (1.) is useless without (2.). I do think that if (1.) is adopted then eventually, someone will implement (2.), but I’d expect it to be opt-in.
My increasing opinion of -Wsign-conversion is that it is too noisy to be useful in a language like C that clearly wasn’t designed for it, and that it often makes existing code worse by forcing programmers to change working code (maybe breaking it in the progress), often by adding redundant casts which make the code less readable and sacrifice type-safety. Still, a lot of people seem to like it.
Then the initial null-initialization can no longer be performed, so a programmer who tries to minimize code changes caused by keeping the pointer non-null will be tempted to simply drop the initializer:
…
Then if one of the branches forgets to initialize the pointer, you’ll end up with a garbage pointer that’s much more terrifying than a null pointer.
I already see this behaviour as a result of warnings about values which are initialized but never used. I can’t remember whether they are generated by Coverity, a compiler, or some other tool. I think this is a case of “pick your poison”. If you use a tool which produces such warnings, and choose to fix them in that way, you’d better be sure it also warns you about garbage values.
Separately, while it may be possible to annotate all your project code, I don’t see how you can easily extend this contract on third-party code that you’re using.
The project that I used for prototyping uses very little third-party code. There’s nothing stopping you from passing a null pointer constant into third-party code directly, because I deliberately did not respecify null. In other cases, a wrapper might be required:
Here’s a wrapper for free which works for C or C++:
static inline void my_free(_Optional void *ptr)
{
free((void *)ptr);
}
Here’s a nicer (albeit less efficient, but probably not measurably) wrapper which would be sufficient for C:
static inline void my_free(_Optional void *ptr)
{
if (ptr)
{
free(&*ptr);
}
}
You might find this horrifying, but it is one small function definition in a huge project, in which all the rest of the code can benefit from tracking null pointers as part of the type.
That’s enough for one reply.
You’re aware free is defined to work on NULL pointers?..
You’re aware free is defined to work on NULL pointers?..
Have you ever tried calling free with a pointer to const? With the exception of the address-of operator, _Optional behaves exactly like const. If it didn’t, I never would have proposed it.
So just cast it away if your standard library headers aren’t _Optional aware? Though I do wonder how viable code is going to be if your standard library isn’t sufficiently annotated. Adding the branch adds overhead that isn’t necessary.
So I strongly suspect that this approach will need massive quality of life improvements which will ultimately make it much more in line with existing solutions, which were the root cause of your frustration in the first place.
That’s why I spent my Christmas holidays using the proposed feature on a large scale. I liked it before when it was only theoretical, and I still like it now.
It’s designed to be trivial to implement, and intuitive to use. If it isn’t nice to use, then fine, I wasted my time. What I do find frustrating is the number of people picking theoretical holes and proposing alternative syntax without ever having actually used the feature. It makes all the effort I put into producing a working prototype feel like a waste of time.
Aaron previously wrote:
having an implementation in hand to play with would go a long ways towards proving the concept is implementable and allowing us to see what the ergonomics of the feature are in practice.
However the availability of an (incredibly simple) implementation doesn’t seem to have made any difference in practice.
Well, if it was a property of the pointed-to object, then it would also make sense to write the same with zero stars:
const int i; // i is an int that _is_ stored in read-only memory volatile int j; // j is an int that _is_ stored in shared memory _Optional int k; // k is an int for which no storage is allocated???
You misquoted my comments from the original article:
_Optional int *k; // *k is an int for which no storage may be allocated
Note that it says may be allocated, not is allocated. Originally, I intended to allow such declarations; ⚙ D142738 Warn if _Optional used at top-level of decl disallows them. I put every part of my proposal in a different commit to allow people to mix-and-match.
If your declaration _Optional int k were allowed, then the address of k would have type int *. I actually like the symmetry of that: it tickles me, it keeps the language simple, and makes it easier to discover the rule for & that is applied in every other expression.
It’s currently disallowed because Aaron previously wrote:
I’d still recommend exploring the design space of making
_Optionala qualifier that is grammatically only allowed on a pointer. This removes several problems with the feature, such as people trying to write_Optional int i
and I was willing to sacrifice some of the orthogonality of my proposal to try to save the rest of it. I’m beginning to regret it though, because a lot of people seem to be jumping to the wrong conclusion that just because _Optional int i isn’t allowed, that means optional objects cannot exist and therefore the qualifier should be removed implicitly by every dereference operator instead of by &.
Given that this doesn’t make sense for your
_Optionalqualifier,
Arguably it does make sense – see above.
it sounds to me as if you just introduced a different way to spell a pointer qualifier (“let’s put it before the
*, not after the*”) without any substantial differences to semantics.
Sorry but I think you’ve completely misunderstood my intent. The reason for the pretentious ‘Philosophical underpinning’ section at the top of my proposal was to explain things like why I didn’t propose a keyword with new semantics that are arbitrarily divorced from the language’s syntax.
_Optional appears before the * specifically in order that it doesn’t have to be handled differently from const or volatile. The syntax is chosen in order to give the desired semantics for every existing type of statement.
Have you looked into using Clang’s existing nullability attributes? I know you said you use gcc, but it might be the case that gcc maintainers are open to supporting them.
First of all, I don’t think this specific problem even needs an in-code solution at all. Instead, the analysis that causes the warning to appear near
strcmpcan be made smart enough to recognize thats1ands2are never null at this point.
That is true, but I one of the axioms at start of my proposal was that it should be ‘(relatively) easy to create a compiler for [C]’. I later postulated that improved null safety does not require path-sensitive analysis and mentioned some compilers which are not huge, complex and resource-hungry, but still perform a useful job of compiling C programs. I believe it also simplifies analysis if only (syntactic) dereferences need to be checked.
You need such smarts in the analysis anyway, to cover another very important case:
int foo(_Optional const char *s1, _Optional const char *s2) { if (...) { return 0; } assert(s1); assert(s2); return strcmp(s1, s2); }
I can only guess that the if condition you have elided is something which allows the programmer to assume that neither s1 nor s2 is null, despite not explicitly checking for that. From my point of view, the assertions are irrelevant, since I’m assuming they are not checked in release builds.
Honestly, I would be OK with forcing the programmer to check for null values of s1 and s2 in that scenario. It seems a bit silly to create an interface which explicitly allows those values to be null but does not handle the consequences (not even by casting away the qualifier).
Now, obviously, sometimes you really need a “force-unwrap” operator to indicate that you’re sure the pointer can’t be null here. In this case “easy to type” isn’t necessarily valuable; say, Rust chose the syntax
.unwrap()which is designed to catch the eye, be easy to notice and audit.
Sorry but I don’t see why anything needs to be built-in. This looks self-explanatory to me:
_Optional int *x = ...;
assert(x);
int *y = (int *)x;
I don’t expect it to be needed very often, so I see your proposals as a ‘nice-to-have’. It would also be nice to have an optional_cast for use in C++ code, but that wasn’t the language I was mainly concerned with.
I don’t see it as fundamentally different from the following commonplace code:
int x = ...;
assert(x >= 0);
unsigned int x = (int)x;
So like I said in the other thread, I think this attribute doesn’t need to be checked by the static analyzer. The contract behind your attribute can be much simpler, fully resolved with either purely syntactic analysis or with very basic flow-sensitive analysis.
It was one of my design goals that purely syntactic analysis should be sufficient. I used that for all my early testing. However, if you think you can implement basic flow-sensitive analysis in the compiler without requiring use of the static analyzer, I’m very interested in that. I wouldn’t know where to start.
The static analyzer can take advantage of it. You can introduce a warning about any unchecked dereference of the _Optional pointer.
I already implemented that, and my new checks catch a lot of undefined behaviour that was previously ignored. I really like that.
The easiest way to introduce such warning is to perform a state split every time the pointer is encountered: in one state the pointer is null, in the other state it’s non-null. Then the null case simply becomes a path that the analyzer has to explore.
I wanted to do that, and even had an attempt, but it didn’t seem to be necessary to make my prototype useful and I didn’t want my wife to divorce me during my paternity leave. Again, if you think you can do this, then that would be wonderful.
I considered specifying as part of my paper how static analysis should work but deliberately left the wording vague in the expectation that different implementions would diverge. If I’m honest, I think this is the biggest weakness of my paper, not the endless arguments over syntax. However, I take solace from the fact that different toolchains already generate different warnings (or none) for the same code.
Maybe there’s still room in the analyzer to warn about invalid force-unwraps, but most of such warnings would be about potential execution paths that the developer has just explicitly said aren’t there, aka false positives.
That sounds like a bad idea to me, and it seems you agree. I want to maintain a strong distinction between verifiable unwraps (&*s) and force-unwraps ((int *)s).
Thank you for your thoughtful comments.
The syntax I had in mind was something like
memcpy(exclusive void *dst, exclusive const void *src, size_t n);
I explained in my paper why that would give better semantics for compatibility of declarations, although the presumed semantics for assignment could be less safe.
You could also compare the simplicity of my patch to check that _Optional isn’t used at top-level (⚙ D142738 Warn if _Optional used at top-level of decl) with existing code to check that restrict isn’t used at bottom-level, which has to check whether the type is a pointer (of any variety) in BuildQualifiedType for every level of GetFullTypeForDeclarator (which walks the DeclTypeInfo backwards) as well as in GetDeclSpecTypeForDeclarator.
I’ve also seen it argued that restrict should be the default, and that aliasing pointers should instead be explicitly qualified. Just like pointers that can be null are in a minority, so are pointers that can alias, so I can see value in that alternative universe.
In that case, this would apply:
memmove(aliased void *dst, aliased const void *src, size_t n);
This would have resolved the conflict between the qualifier-enables-optimisation behaviour of restrict with the qualifier-disables-optimisation behaviour of volatile. It’s all ancient history though.
It differs from _Optional, whether documented or not. Do you think that _Nullable semantically differs from const? I assume so. The code to implement the nullability qualifiers in Clang is vastly more complex than the minor additions I made for _Optional (including the static analyzer), yet the user experience is worse in every way:
_Nullable pointers generate no warning._Nullable to _Nonnull generate no warning unless the user specifies -Wnullable-to-nonnull-conversion (not sure how they are meant to know about that)._Nullable to unqualified generate no warning at all, ever._Nullable function pointer generate no warning._Nullable qualified arguments.There’s another thread about those bugs: Nullability analyzer doesn't seem to work (and how to fix it)
In contrast, _Optional has exactly the same behaviour as const in every context except static analysis, where
_Optional do generate a warning, even if it’s only a syntactic dereference. This catches a lot of undefined behaviour._Optional functions via pointers do generate a warning.I’m not sure what your point is. The opinion exists, whether you agree with it or not.
I have (anecdotal) evidence that their operation, to the extent that they work at all, was so obscure to me and my colleagues that I didn’t understand it until I began browsing the source code of Clang.
That’s probably because _Nonnull is the usual usage of pointers in the C language (which is the whole point of my proposal to qualify only the opposite case) and _Nullable appears to be mostly broken/useless.
I can’t really help if you don’t see a problem with the public interface of a module diverging arbitrarily from its actual implementation in ways that the compiler cannot verify.
The difference is that the type checking and static analysis using _Optional would actually work, the implementation in the compiler would be orders of magnitude (10? 20 times?) simpler, and it would provide a high degree of null pointer safety even in a compiler that performs no path-sensitive analysis.
I find it easy to explain: the address of an object is never null. Every C programmer already knows that.
Not sure what you meant by that.
I’d hate that, but sure, go ahead 
I didn’t propose any change to the standard library headers because function signatures have to be backward-compatible. Existing code which uses the address of free() and expects it to have the signature void free(void *) would fail to compile if the signature were instead void free(_Optional void *). The most obvious example is when free is used as a callback function.
As with every other aspect of my proposal, this could be worked around by defining _Optional as an empty macro when invoking the compiler (i.e. in the Makefile or equivalent), but a better idea would be to improve the rules for compatibility of function signatures. I don’t feel like submitting another paper to do that right now.
I’d like to see evidence that it makes any measurable difference. The efficiency of executing modern software is almost entirely bounded by memory access. Untaken branches could increase instruction cache usage, but I still doubt the difference would be measurable unless every other function call were free() and that function was also inlined.