RFC: Nullability qualifiers

Clang Frontend
21

OK. What would be the best way to detect if Apple clang supports _Nonnull or
only __nonnull though.

I cannot speak for how Apple's Clang works in this regard, but perhaps Doug can.

~Aaron

22

Apple please implement __has_feature(nullability) in clang for Xcode 7 release. :slight_smile:

23

I figured out my issue. I was compiling with -std=c++14 and the nullability and assume_nonnull features are only enabled for ObjC and GNU mode. Why are these only supported in GNU mode? I thought GNU mode was only for features that contradict the standard. How does this feature contradict the standard given that the names are double and single underscore prefixed? I would rather not have to compile my code in GNU mode just to enable nullability. I can check with __has_extension() but at least Apple headers seem to only use __has_feature so the checks there won’t be enabled when not compiling in GNU mode. If the consensus is that __has_feature(nullability) should only be enabled for GNU mode, would it make sense to have an f-group flag like -fnullability to enable it for __has_feature when not compiling in GNU mode?

Also I found a bug in clang. __has_extension(assume_nonnull) doesn’t work properly. It is missing from the StringCase at the end of the HasExtension() function in lib/Lex/PPMacroExpansion.cpp. I think it should be there.

24

I found another bug with nullability. Consider this code in a header:

_Pragma(“clang assume_nonnull begin”)
template
inline void f(T x) {}
_Pragma(“clang assume_nonnull end”)

This gives the following bogus warning:

./nullability.h:3:15: warning: pointer is missing a nullability type specifier (__nonnull or __nullable) [-Wnullability-completeness]

25

I figured out my issue. I was compiling with -std=c++14 and the nullability
and assume_nonnull features are only enabled for ObjC and GNU mode. Why are
these only supported in GNU mode? I thought GNU mode was only for features
that contradict the standard. How does this feature contradict the standard
given that the names are double and single underscore prefixed? I would
rather not have to compile my code in GNU mode just to enable nullability. I
can check with __has_extension() but at least Apple headers seem to only use
__has_feature so the checks there won't be enabled when not compiling in GNU
mode. If the consensus is that __has_feature(nullability) should only be
enabled for GNU mode, would it make sense to have an f-group flag like
-fnullability to enable it for __has_feature when not compiling in GNU mode?

Also I found a bug in clang. __has_extension(assume_nonnull) doesn't work
properly. It is missing from the StringCase at the end of the HasExtension()
function in lib/Lex/PPMacroExpansion.cpp. I think it should be there.

__has_feature(assume_nonnull) is the way to test for that feature
(which is also GNU and Obj-C only).

~Aaron

26

__has_extension(nullability) returns true in non-GNU mode, on the other hand,__has_extension(assume_nonnull) returns false in non-GNU mode. Are you saying this difference is by design. If so, why?

27

__has_extension(nullability) returns true in non-GNU mode, on the other
hand,__has_extension(assume_nonnull) returns false in non-GNU mode. Are you
saying this difference is by design. If so, why?

I answered a bit too early in the morning. :wink: I forgot that
__has_extension inherits functionality from __has_feature. So
assume_nonnull should be true with either __has_feature or
__has_extension in GNU or ObjC mode, but is currently false in other
modes. You are correct that nullability is a bit different, and I'm
not certain why. __has_feature(nullability) will return true for GNU
and ObjC mode. __has_extension(nullability) will always return true.

I am not certain whether this is by design or is a bug, but perhaps
Doug can explain. (I'm also a bit curious as to why GNU mode is
required.)

~Aaron

28

Just to clarify…

__has_extension(nullability) always returning true is exactly what I would expect from reading the clang documentation. I would also expect that __has_extension(assume_nonnull) always returns true (which it currently doesn’t).

I am hoping someone can explain why GNU mode is required for __has_feature(nullability) and __has_feature(assume_nonnull) to return true.

29

Just to clarify…

__has_extension(nullability) always returning true is exactly what I would expect from reading the clang documentation. I would also expect that __has_extension(assume_nonnull) always returns true (which it currently doesn’t).

__has_extension(assume_nonnull) should always be true; fixed in r240969.

I am hoping someone can explain why GNU mode is required for __has_feature(nullability) and __has_feature(assume_nonnull) to return true.

GNU mode was meant to capture “the default behavior”, and gets disabled when one specifies stricter conformance to the language standard. Rather, nullability is an extension that is always available, and only a “standardized” language feature in Objective-C (where Clang is essentially the standard). In retrospect, we shouldn’t straddle the fence this way with GNUMode: either it’s an always-on feature (__has_feature and __has_extension are always true for both) or it’s only a language feature in Objective-C (__has_feature is true only in Objective-C mode, __has_extension is always true). I tend to prefer the former.

  • Doug
30

Just to clarify…

__has_extension(nullability) always returning true is exactly what I would expect from reading the clang documentation. I would also expect that __has_extension(assume_nonnull) always returns true (which it currently doesn’t).

__has_extension(assume_nonnull) should always be true; fixed in r240969.

I am hoping someone can explain why GNU mode is required for __has_feature(nullability) and __has_feature(assume_nonnull) to return true.

GNU mode was meant to capture “the default behavior”, and gets disabled when one specifies stricter conformance to the language standard. Rather, nullability is an extension that is always available, and only a “standardized” language feature in Objective-C (where Clang is essentially the standard). In retrospect, we shouldn’t straddle the fence this way with GNUMode: either it’s an always-on feature (__has_feature and __has_extension are always true for both) or it’s only a language feature in Objective-C (__has_feature is true only in Objective-C mode, __has_extension is always true). I tend to prefer the former.

r240976 makes these always-true.

  • Doug
31

Hi Doug,
Thanks for your changes. I think the choice you made makes the most sense.

Can you look at the nullability-completeness bug with templated types? Or should I make a bug report?

32

Hi Doug,
Thanks for your changes. I think the choice you made makes the most sense.

Can you look at the nullability-completeness bug with templated types? Or should I make a bug report?

Please file a bug report. It helps us track fixes.

  • Doug
33

Filed bug 23987.

34

Hi, I have only just discovered this thread and it looks very interesting. Was a type qualifier added to Clang or submitted for possible standardization in C23?

In a recent version of Xcode, Apple introduced an extension to
C/C++/Objective-C that expresses the nullability of pointers in the type
system via new nullability qualifiers . Nullability qualifiers express
nullability as part of the declaration of strchr [2]:

__nullable char *strchr(__nonnull const char *s, int c);

I think this is the correct place to put the qualifier, for reasons explained below.

Coincidentally, I was thinking along very similar lines on Friday evening. I was considering the practicality of implementing something like Python’s Optional[] type-checking in C. I didn’t consider name-mangling or C++. (Apologies for the didactic style – it helped me get my thoughts coherent):

Type qualifiers are restrictions (or promises). A restriction on a pointer target type can be added implicitly but must be explicitly cast away; a restriction on any other type (including a pointer itself) is lost when the value is copied by assignment. This will be important later.

  • const (‘not-writable’) is a usage restriction that the value of an object may not be modified. The compiler refuses to compile code which violates this.
  • volatile (‘not-cacheable’) is a usage restriction that the value of an object may not be cached in registers. The compiler automatically generates code which complies.
  • restrict (‘not-aliased’) is a usage restriction that the value of an object may not be accessed except via the specified pointer. This is actually an anti-restriction because it frees the compiler to generate code which assumes no aliasing. It would have been more consistent if compilers assumed by default that pointers do not alias (just as they assume that most objects are not volatile). A fictional alternative alias qualifier could force generation of safe code.

volatile and const qualifiers can be applied to every object type, whereas restrict only applies to pointers (which is a bit ugly). Because restrict does not apply to the pointer target type, it does not need to be explicitly cast away and therefore is not type-safe. Unfortunately a nullable or notnull qualifier would fall into the same category as restrict (e.g. int *notnull x would be a promise to the compiler that the target of pointer x exists, but could also be implicitly converted to int *y).

  • int *nullable could be a usage restriction that a pointer may not be dereferenced or used for arithmetic without first checking it is not null. The compiler cannot implement such checking itself because it affects control flow (or else would have UB). Static analysis tools could check for execution paths on which such checks are absent, but this checking could already be done because any pointer in C can be null. Effectively every object reference in C is already Optional[...] by default.

  • int *notnull could be a restriction that the value of a pointer is not null (like a C++ reference). This looks like a more plausible guarantee because it doesn’t require any runtime checking. When generating code for pointers with this qualifier, the compiler could assume that they will never equal null (like it assumes restrict pointers will never alias). For this qualifier to improve type-safety, the compiler must check that ordinary pointer types (including (void*)0) are not assigned to a pointer declared as notnull. That would break the existing pattern for type qualifiers, which is that an object of unqualified type may be assigned to an object of the equivalent qualified type. restrict allows this implicit conversion, which is one reason it is dangerous.

  • int optional * could be a pointer target type qualifier indicating that an object may not exist (e.g. int optional *y; makes sense, but int optional z; does not). Taking the address of an optional object would generate a pointer-to-optional, and this qualifier would need to be explicitly cast away (as for a pointer-to-const). This qualifier could be added to some existing interfaces (e.g. free(void optional *) wouldn’t require the pointer target to be optional any more than strcmp(const char *, const char *) requires its pointer targets to be const), but the effect of adding optional to the pointer target returned by malloc (which is the main source of null pointers) would be to prevent a huge amount of existing code from compiling. The only way to introduce it in a backward-compatible way would be to add ‘safe’ variants of malloc, calloc, fopen and many other standard functions.

  • int nonopt * could be a pointer target type qualifier indicating that an object must exist (e.g. int nonopt *y; makes sense, but int nonopt z; does not). However, this notion isn’t compatible with the fact that C adds pointer target type qualifiers implicitly on assignment. In practice, therefore, int nonopt *y would mean that y could be a pointer that may not be null (like any existing pointer declaration), just as int const *x means that x could be the address of an object which may not be modified. This information is meaningless.

Of all the proposed extensions, optional seems to be the only one which would make code more type-safe, and therefore the only one of potential interest to me. notnull could improve code generation, at the cost of introducing hard-to-debug errors (like restrict).

Obviously, optional isn’t a reserved keyword, so I envisage a header file analogous to <stdbool.h> in which supporting compilers #define optional _Optional. Compilers that don’t support optional could instead simply #define optional as nothing.

35

Yes, this qualifier was added to Clang (in Clang 3.7), and you can see the documentation for it at: Attributes in Clang — Clang 18.0.0git documentation. This functionality was not proposed to WG14 as far as I’m aware (and it’s definitely too late for C23 which is going to CD ballot shortly), but would be appropriate to propose for a future revision of C.

36

Hi Aaron, thanks for your quick reply.

I did some experimentation with __Nullable in Compiler Explorer, and it doesn’t seem to behave like I would expect a type qualifier to behave. Presumably this is because it is powered by black magic rather than the standard type system. This also makes me doubt that it would be effective across separate compilation units.

For example:

  • char *_Nullable can be implicitly assigned to char *. Presumably this is because the alternative would be to “break the existing pattern for type qualifiers, which is that an object of unqualified type may be assigned to an object of the equivalent qualified type”, as I wrote in my original post. Nevertheless, this isn’t desirable from my point of view, and reflects the fact that __Nullable applies to the pointer type, not the pointer target type.
  • Dereferencing a pointer deemed ‘nullable’ by the compiler generates a warning even if the qualifier is explicitly removed by a type cast. This is very magical and does not reflect the behaviour of any other pointer target type qualifier like const or volatile.
37

Is presumably the root cause of the behaviour I’m seeing. I’m not sure that such a feature would qualify for adoption as part of the C standard. It also seems to require a complex inference to be of any use (and even then, I’m not convinced, without access by the static analyser to the full source code of a program) whereas my proposal was relatively simple, as it has almost exactly the same syntax and semantics as const.

38

See previous discussion ISO C3X proposal: nonnull qualifier

1 Like