using declaration questions

Hi Doug,

I’ve been working on using declarations a bit over the last week, amidst continued fire-fighting elsewhere, butI thought I’d better address some parts of it in pieces, rather than in a big unwieldy chunk. Rather than a patch, I’ll just list relevent pieces to discuss, as I have questions about the code. I hope it formats in a readable way here.

We’ll probably want to store more location information, e.g., the source range covering the nested-name-specifier, (e.g., “N::m::”), the source location of the target declaration’s name, and the source location of the “using” itself. Also, how about keeping track of the NestedNameSpecifier used to refer to the target declaration? It’s good for pretty-printing, and will also be needed when the using directive occurs within a template and the nested-name-specifier is dependent.

I was a litte fuzzy about the exact meaning of these, but here’s my best guess. First the new renamed UsingDecl:

/// UsingDecl - Represents C++ using-directive. For example:
/// using someNameSpace::someIdentifier;

class UsingDecl : public NamedDecl {
/// \brief The source range that covers the nested-name-specifier
/// preceding the namespace name.
SourceRange NestedNameRange;
/// \brief The source location of the target declaration name.
SourceLocation TargetNameLocation;
/// \brief The source location of the “using” location itself.
SourceLocation UsingLocation;
/// \brief Target declaration.
NamedDecl* TargetDecl;
/// \brief Target declaration.
NestedNameSpecifier* TargetNestedNameDecl;
// Had ‘typename’ keyword.
bool IsTypeName;
UsingDecl(DeclContext DC, SourceLocation L, SourceRange NNR,
SourceLocation TargetNL, SourceLocation UL, NamedDecl
Target,
NestedNameSpecifier* TargetNNS, bool IsTypeNameArg)
: NamedDecl(Decl::Using, DC, L, Target->getDeclName()),
NestedNameRange(NNR), TargetNameLocation(TargetNL),
UsingLocation(UL), TargetDecl(Target),
TargetNestedNameDecl(TargetNNS), IsTypeName(IsTypeNameArg) { }
public:
/// \brief Returns the source range that covers the nested-name-specifier
/// preceding the namespace name.
SourceRange getNestedNameRange() { return(NestedNameRange); }
/// \brief Returns the source location of the target declaration name.
SourceLocation getTargetNameLocation() { return (TargetNameLocation); }
/// \brief Returns the source location of the “using” location itself.
SourceLocation getUsingLocation() { return(UsingLocation); }
/// \brief getTargetDecl - Returns target specified by using-decl.
NamedDecl getTargetDecl() { return(TargetDecl); }
/// \brief Get target nested name declaration.
NestedNameSpecifier
getTargetNestedNameDecl() { return(TargetNestedNameDecl); }
/// \brief isTypeName - Return true if using decl had ‘typename’.
bool isTypeName() const { return(IsTypeName); }

static UsingDecl Create(ASTContext &C, DeclContext DC,
SourceLocation L, SourceRange NNR, SourceLocation TargetNL,
SourceLocation UL, NamedDecl
Target,
NestedNameSpecifier
TargetNNS, bool IsTypeNameArg);

static bool classof(const Decl *D) {
return D->getKind() == Decl::Using;
}
static bool classof(const UsingDecl *D) { return true; }
};

Regarding the later comment:

Oh, and we might even consider storing the list of declarations that the using declaration refers to, because we need to model the semantics of paragraph 11:

The entity declared by a using-declaration shall be known in the context using it according to its definition
at the point of the using-declaration . Definitions added to the namespace after the using-declaration are not
considered when a use of the name is made.

I’m not sure what list of declarations is needed. If later definitions appear, it seems that a look up will find the using declaration first, and return the refered-to declaration. Or does this have something to do with overloading?

The additional source locations are handled in ActOnUsingDeclaration like:

UsingAlias = UsingDecl::Create(Context, CurContext, IdentLoc, SS.getRange(),
NS->getLocation(), UsingLoc, NS,
static_cast<NestedNameSpecifier *>(SS.getScopeRep()),
IsTypeName);

Does this seem right?

Lastly, do you think Sema::LookupParsedName is the right place to do the substitution for non-redeclaration lookups? Or should it be lower in LookupName, with more conditions to exclude certain lookup types?

I’ll be experimenting with this next, as well as writing a bigger test, to ring out some of the trickier cases.

-John

Overloading is precisely the issue: a using declaration can refer to
multiple declarations.

-Eli

Not sure how forward-looking you want to be with this, but I have recently taken a look at using declarations myself with a view to the C++0x grammar.

Putting aside the concept related bits that Doug might explain better, the two main new constructs are alias declarations and inheriting constructors.

You can distinguish an alias declaration because it is not allowed a nested-name-specifier, and the identifier will be followed by an ‘=’.

using my_type = typename some_template::type;

Inheriting constructor declarations are only allowed inside class (or class template) definitions, and simply name a direct base-class twice:

using base::base;

The effect here, though, is not to make a name available for name lookup but to declare a family of similar constructors directly in the derived class.

I suspect there is little to cater for in terms of supporting inheriting constructors in the future, but it might be useful to plan for the alias declaration.

AlisdairM

Hi Doug,

I’ve been working on using declarations a bit over the last week, amidst continued fire-fighting elsewhere, butI thought I’d better address some parts of it in pieces, rather than in a big unwieldy chunk. Rather than a patch, I’ll just list relevent pieces to discuss, as I have questions about the code. I hope it formats in a readable way here.

We’ll probably want to store more location information, e.g., the source range covering the nested-name-specifier, (e.g., “N::m::”), the source location of the target declaration’s name, and the source location of the “using” itself. Also, how about keeping track of the NestedNameSpecifier used to refer to the target declaration? It’s good for pretty-printing, and will also be needed when the using directive occurs within a template and the nested-name-specifier is dependent.

I was a litte fuzzy about the exact meaning of these, but here’s my best guess. First the new renamed UsingDecl:

/// UsingDecl - Represents C++ using-directive. For example:
/// using someNameSpace::someIdentifier;

class UsingDecl : public NamedDecl {
/// \brief The source range that covers the nested-name-specifier
/// preceding the namespace name.
SourceRange NestedNameRange;

Typo: “preceding the declaration name.”

/// \brief The source location of the target declaration name.
SourceLocation TargetNameLocation;
/// \brief The source location of the “using” location itself.
SourceLocation UsingLocation;
/// \brief Target declaration.
NamedDecl* TargetDecl;
/// \brief Target declaration.
NestedNameSpecifier* TargetNestedNameDecl;

This is often called the “qualifier” of a qualified name (in the AST).

// Had ‘typename’ keyword.
bool IsTypeName;
UsingDecl(DeclContext DC, SourceLocation L, SourceRange NNR,
SourceLocation TargetNL, SourceLocation UL, NamedDecl
Target,
NestedNameSpecifier* TargetNNS, bool IsTypeNameArg)
: NamedDecl(Decl::Using, DC, L, Target->getDeclName()),
NestedNameRange(NNR), TargetNameLocation(TargetNL),
UsingLocation(UL), TargetDecl(Target),
TargetNestedNameDecl(TargetNNS), IsTypeName(IsTypeNameArg) { }
public:
/// \brief Returns the source range that covers the nested-name-specifier
/// preceding the namespace name.
SourceRange getNestedNameRange() { return(NestedNameRange); }
/// \brief Returns the source location of the target declaration name.
SourceLocation getTargetNameLocation() { return (TargetNameLocation); }
/// \brief Returns the source location of the “using” location itself.
SourceLocation getUsingLocation() { return(UsingLocation); }
/// \brief getTargetDecl - Returns target specified by using-decl.
NamedDecl getTargetDecl() { return(TargetDecl); }
/// \brief Get target nested name declaration.
NestedNameSpecifier
getTargetNestedNameDecl() { return(TargetNestedNameDecl); }
/// \brief isTypeName - Return true if using decl had ‘typename’.
bool isTypeName() const { return(IsTypeName); }

static UsingDecl Create(ASTContext &C, DeclContext DC,
SourceLocation L, SourceRange NNR, SourceLocation TargetNL,
SourceLocation UL, NamedDecl
Target,
NestedNameSpecifier
TargetNNS, bool IsTypeNameArg);

static bool classof(const Decl *D) {
return D->getKind() == Decl::Using;
}
static bool classof(const UsingDecl *D) { return true; }
};

This looks right to me.

Regarding the later comment:

Oh, and we might even consider storing the list of declarations that the using declaration refers to, because we need to model the semantics of paragraph 11:

The entity declared by a using-declaration shall be known in the context using it according to its definition
at the point of the using-declaration . Definitions added to the namespace after the using-declaration are not
considered when a use of the name is made.

I’m not sure what list of declarations is needed. If later definitions appear, it seems that a look up will find the using declaration first, and return the refered-to declaration. Or does this have something to do with overloading?

Yes, it’s for overloading. It will work as-is, because TargetDecl can be an OverloadedFunctionDecl, but in the longer term I’d like to eliminate OverloadedFunctionDecl.

The additional source locations are handled in ActOnUsingDeclaration like:

UsingAlias = UsingDecl::Create(Context, CurContext, IdentLoc, SS.getRange(),
NS->getLocation(), UsingLoc, NS,
static_cast<NestedNameSpecifier *>(SS.getScopeRep()),
IsTypeName);

Does this seem right?

Yes.

Lastly, do you think Sema::LookupParsedName is the right place to do the substitution for non-redeclaration lookups? Or should it be lower in LookupName, with more conditions to exclude certain lookup types?

I’ll be experimenting with this next, as well as writing a bigger test, to ring out some of the trickier cases.

Okay. If you’d like to submit a patch with just parsing + AST, then tackle semantic analysis as a separate step, that’d be fine.

  • Doug

Generally, I suggest that you post patches to the list, because there are others who might also provide reviews. Some comments follow.

Sorry Doug, I copied both you and the list, which is redundant, so I’ll just use the list from now on.

Index: tools/clang/include/clang/Basic/Builtins.h

===================================================================
— tools/clang/include/clang/Basic/Builtins.h (revision 73735)
+++ tools/clang/include/clang/Basic/Builtins.h (working copy)
@@ -31,6 +31,7 @@

enum ID {
NotBuiltin = 0, // This is not a builtin function.
#define BUILTIN(ID, TYPE, ATTRS) BI##ID,
+#undef alloca
#include “clang/Basic/Builtins.def”
FirstTSBuiltin
};

I thought this build issue was fixed. Do you still need this?

Actually, I didn’t see my prior change in the update, but for some reason, with my original change, I still was getting the error, even after a full rebuild. Perhaps it was a weirdness with Visual Studio. But moving the undefine up into the header fixed it, again. Do you think I should redefine it (#define abort _abort) for Windows after the Buildtins.def include?

Index: tools/clang/lib/AST/DeclBase.cpp

— tools/clang/lib/AST/DeclBase.cpp (revision 73735)
+++ tools/clang/lib/AST/DeclBase.cpp (working copy)
@@ -164,6 +164,7 @@

case ParmVar:
case OriginalParmVar:
case NonTypeTemplateParm:

  • case Using:
    case ObjCMethod:
    case ObjCContainer:
    case ObjCCategory:

This puts all using declarations into the ordinary namespace. However, what happens if the using declaration refers to something not in the ordinary namespace, e.g.,

namespace N {
struct X;
}

using N::X;
void X(int); // okay

struct X *x; // okay

I guess my expectation was that UsingDecl would bitwise-OR the identifier namespace values of all of the things that it points to.

Sorry, I blindly put that in not understanding it while searching for places where decl types were used. I’m afraid I still don’t understand it, so I’ll have to study it some more as part of working on the semantics, unless you have an easy fix. I’ve left what I had in for now, to keep the test file working.

  • // Parse namespace-name.
  • if (SS.isInvalid() || Tok.isNot(tok::identifier)) {
  • Diag(Tok, diag::err_expected_namespace_or_type_name);
  • // If there was invalid namespace name, skip to end of decl, and eat ‘;’.
  • SkipUntil(tok::semi);
  • return DeclPtrTy();
  • }

“expected namespace or type name” seems like the wrong diagnostic. I think we should separate the two cases:

  • If the scope specifier is invalid, we’ve already emitted an error so we can just return.

  • If the token following the scope specifier is not an identifier, we can say something like “expected an identifier in using directive”. It would also be really great if we detected the case of a template-id (e.g., “using std::vector;”) specifically (C++ [namespace.udecl]p5), because it’s such an easy error to make. This is a case where we can provide a very specific diagnostic.

I’ve taken a stab at fixing this in the new patch.

  • // Lookup target name.
  • LookupResult R = LookupParsedName(S, &SS, TargetName,
  • LookupOrdinaryName, false);
  • if (NamedDecl *NS = R) {
  • if (IsTypeName && !isa(NS)) {
  • Diag(IdentLoc, diag::err_using_typename_non_type);
  • return DeclPtrTy();
  • }

Good, good. Once we’ve emitted the error, I don’t think we have to return, because we can recover from this particular error easily.

So just delete the return statement? I’ve added that to the new patch.

Index: tools/clang/lib/Sema/SemaLookup.cpp

— tools/clang/lib/Sema/SemaLookup.cpp (revision 73735)
+++ tools/clang/lib/Sema/SemaLookup.cpp (working copy)
@@ -1151,8 +1151,13 @@

Name, NameKind, RedeclarationOnly);
}

  • return LookupName(S, Name, NameKind, RedeclarationOnly,
  • AllowBuiltinCreation, Loc);
  • LookupResult result(LookupName(S, Name, NameKind, RedeclarationOnly,
  • AllowBuiltinCreation, Loc));
  • UsingDecl* usingDecl;
  • if (!RedeclarationOnly && (bool)result &&
  • ((usingDecl = dyn_cast(result.getAsDecl())) != NULL))
  • return(LookupResult::CreateLookupResult(Context, usingDecl->getTargetDecl()));
  • return(result);
    }

This is only one of many places where we would have to perform the UsingDecl → target declaration mapping. I think it would be much easier to move this actual mapping into CreateLookupResult (or some helper that we use instead of CreateLookupResult). That way, there’s only one point where we have to cope with using declarations.

Okay. I’ll work on this after this patch.

We’re definitely getting closer. My comments about the parsing side of things were pretty trivial; if you want to make those little changes first, I can commit that part of your patch to the tree so you can focus on semantics (which are significantly tougher).

  • Doug

Thanks, Doug. I’ve enclosed a new patch, usingdecl3.patch. I’ve also added your examples to the test file, the last part of which is for later work. I left in a couple of the old changes to SemaDecl to keep the test file working, but you can ignore them if you want. Otherwise I’ll take them out when I have the substitution working.

I hope my working on Clang is working out for you guys, and not taking more hand-holding than it is worth. I fear I might be in a bit over my head, but I’d definitely like to and help out, if I can.

usingdecl3.patch (17.9 KB)

cxx-using-declaration.cpp (829 Bytes)

Generally, I suggest that you post patches to the list, because there are others who might also provide reviews. Some comments follow.

Sorry Doug, I copied both you and the list, which is redundant, so I'll just use the list from now on.

Thanks. All of the Clang developers follow this list closely.

Index: tools/clang/include/clang/Basic/Builtins.h

===================================================================
--- tools/clang/include/clang/Basic/Builtins.h (revision 73735)
+++ tools/clang/include/clang/Basic/Builtins.h (working copy)
@@ -31,6 +31,7 @@

enum ID {
  NotBuiltin = 0, // This is not a builtin function.
#define BUILTIN(ID, TYPE, ATTRS) BI##ID,
+#undef alloca
#include "clang/Basic/Builtins.def"
  FirstTSBuiltin
};

I thought this build issue was fixed. Do you still need this?

Actually, I didn't see my prior change in the update, but for some reason, with my original change, I still was getting the error, even after a full rebuild. Perhaps it was a weirdness with Visual Studio. But moving the undefine up into the header fixed it, again. Do you think I should redefine it (#define abort _abort) for Windows after the Buildtins.def include?

Ugh. What a pain. If we need this to build on Windows, that's fine, but I'd like to to have the appropriate #ifdef's to keep this a Windows-only hack.

Index: tools/clang/lib/AST/DeclBase.cpp

--- tools/clang/lib/AST/DeclBase.cpp (revision 73735)
+++ tools/clang/lib/AST/DeclBase.cpp (working copy)
@@ -164,6 +164,7 @@

    case ParmVar:
    case OriginalParmVar:
    case NonTypeTemplateParm:
+ case Using:
    case ObjCMethod:
    case ObjCContainer:
    case ObjCCategory:

This puts all using declarations into the ordinary namespace. However, what happens if the using declaration refers to something not in the ordinary namespace, e.g.,

namespace N {
   struct X;
}

using N::X;
void X(int); // okay

struct X *x; // okay

I guess my expectation was that UsingDecl would bitwise-OR the identifier namespace values of all of the things that it points to.

Sorry, I blindly put that in not understanding it while searching for places where decl types were used. I'm afraid I still don't understand it, so I'll have to study it some more as part of working on the semantics, unless you have an easy fix. I've left what I had in for now, to keep the test file working.

The easy fix, which is good enough for now, is to get the target declaration's identifier namespace. Eventually, we'll have to cope with a using declaration that names both a tag and a value; then it'll be more fun :slight_smile:

+ // Parse namespace-name.
+ if (SS.isInvalid() || Tok.isNot(tok::identifier)) {
+ Diag(Tok, diag::err_expected_namespace_or_type_name);
+ // If there was invalid namespace name, skip to end of decl, and eat ';'.
+ SkipUntil(tok::semi);
+ return DeclPtrTy();
+ }

"expected namespace or type name" seems like the wrong diagnostic. I think we should separate the two cases:
       - If the scope specifier is invalid, we've already emitted an error so we can just return.

       - If the token following the scope specifier is not an identifier, we can say something like "expected an identifier in using directive". It would also be really great if we detected the case of a template-id (e.g., "using std::vector<int>;") specifically (C++ [namespace.udecl]p5), because it's such an easy error to make. This is a case where we can provide a very specific diagnostic.
I've taken a stab at fixing this in the new patch.

Looks good.

+ // Lookup target name.
+ LookupResult R = LookupParsedName(S, &SS, TargetName,
+ LookupOrdinaryName, false);
+
+ if (NamedDecl *NS = R) {
+ if (IsTypeName && !isa<TypeDecl>(NS)) {
+ Diag(IdentLoc, diag::err_using_typename_non_type);
+ return DeclPtrTy();
+ }

Good, good. Once we've emitted the error, I don't think we have to return, because we can recover from this particular error easily.
So just delete the return statement? I've added that to the new patch.

That's fine.

Thanks, Doug. I've enclosed a new patch, usingdecl3.patch. I've also added your examples to the test file, the last part of which is for later work. I left in a couple of the old changes to SemaDecl to keep the test file working, but you can ignore them if you want. Otherwise I'll take them out when I have the substitution working.

Did you run the Clang test suite with this patch? I'm seeing several new failures with your patch:

  /Volumes/Data/dgregor/Projects/llvm/tools/clang/test/Parser/cxx-using-directive.cpp
  /Volumes/Data/dgregor/Projects/llvm/tools/clang/test/SemaCXX/member-name-lookup.cpp
  /Volumes/Data/dgregor/Projects/llvm/tools/clang/test/SemaCXX/namespace-alias.cpp
  /Volumes/Data/dgregor/Projects/llvm/tools/clang/test/SemaCXX/using-directive.cpp

I fixed the first one (it was just a changed error message) and the non-semantic issue in the last one. However, I ended up removing the new code in SemaLookup, which was causing the other failures. At a minimum, all existing tests will have to pass before we can put in the name-lookup part of your patch.

I hope my working on Clang is working out for you guys, and not taking more hand-holding than it is worth. I fear I might be in a bit over my head, but I'd definitely like to and help out, if I can.

There's always a ramp-up, especially for C++. We know that :slight_smile:

Some nit-picks follow. I've committed everything except the SemaLookup part; thanks! The commit is here:

  http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20090615/018333.html

+def err_expected_ident_in_using : Error<"expected an identifier in using directive">;
+def err_unexpected_template_inst_in_using : Error<"use of template instantiation in using directive not allowed">;

Please keep lines to <= 80 columns.

And, to be *really* pedantic, the template-id refers to a template specialization, not a template instantiation. I've fixed the wording.

Index: tools/clang/include/clang/Parse/Action.h