I think that’s very reasonable. I apologize that I’ve been caught up in debating the long-term direction, and I haven’t been considering your short-term needs. Even for the long-term direction, there might be critical issues we still need to consider for your use case. I’m happy to keep discussing how/whether the solution I’m proposing works for you.
I will attempt to prepare that patch for phabricator in the next couple of days so you can get a sense of it. From a high-level view within lit’s implementation, I think it’s a conceptually simple extension. It’s just a substitution. The regex to match uses of function substitutions are where it gets a bit subtle in the implementation.
If your D131464 works well for your use case now, and if it’s reasonable to migrate to something else later, my personal opinion is that you shouldn’t wait. Some well placed comments could hopefully warn people that multi-valued substitutions will likely be replaced later, so they should be prepared for that if they want to use them now.
I say don’t wait because I want the DEFINE/REDEFINE series to get a proper review from multiple people who have expressed interest. It’s possible someone will discover it doesn’t do what they had hoped, so we’d need to think about it more. They might have great suggestions that require major changes that would break backward compatibility if we land too early. All this might be quick or might be slow.
You’d also need to provide an initial definition for %{check-std} so it can be REDEFINEd. %{check-std} is a function substitution, so its definitions in python (python pattern and replacement string) are tricky to get right. My function substitution patch will extend DEFINE/REDEFINE to do that for you, but I still need to expose that functionality to lit configurations files so they can easily do the same thing. I don’t think implementing that exposure will be challenging, and I want it too, but I haven’t gotten to it yet.
An alternative approach is to create a lit include feature so you can write out those definitions using DEFINE/REDEFINE in the same way you would write them in an individual test. I also haven’t implemented that yet.
One question for you: do you need to support every possible range within the C++ standards list? That would be a lot of lit substitutions to define. Or is there a specific set of ranges that would suffice and could be extended as needed?
I originally implemented it to use , and ). I then tried to use that in my own downstream test suites, and I quickly ran into many cases where I needed , and ) in my actual arguments. For example, an argument might be a comma-delimited list of FileCheck/-verify prefixes. An argument might contain expected output strings that contained parenthesized elements, such as (null).
I played around with permitting either set of delimiters because one is pretty and the other offers flexibility I sometimes need. However, having two syntactic forms cluttered the implementation, and I kept mixing things up when writing tests.
Ultimately, I decided just %, and %) are the best way to go. I don’t think they’re particularly pretty, but they mean there’s no limit on what can go in an argument. Moreover, they fit the general theme that % is the only special character. The %if syntax ultimately followed the same theme.
But, like anything else, the syntax can be debated.
In my current implementation, you do need to avoid +. The documentation specifies exactly what’s permitted, and lit diagnoses violations. I made it very strict. There’s a simple reason: the name is a pattern applied using python’s re.sub. That’s already how substitutions work in lit. We could implement DEFINE/REDEFINE to escape the name, but what happens when you decide to relocate the name into a lit configuration file? You then have to remember to escape it yourself. I predict people will often forget (I did before I restricted things), substitutions will quietly match text they weren’t intended to, and we’ll be lucky if tests fail. For example, . is a particularly insidious character my implementation doesn’t permit even though I really like the way it looks in names.
They can appear in arguments. See discussion above.
I’m not sure I understand the point you’re making here. Can you explain a bit more?