I have a couple of ideas here. But before I list them I wanted to ask how you guys build LLVM. LLVM doesn't have a native Xcode build, so presumably you're already running CMake once in order to build LLVM for LLDB to link against. Can you expand on this workflow a little?
There is a script in the scripts folder (build-llvm.pl) that gets invoked from the Xcode build of lldb. That checks out (if necessary) and builds llvm/clang. That script used to use the Makefile build but we switched that to cmake a while ago. That script is invoked by one of the lldb Xcode targets. So from Xcode you just build the overall lldb-tool target and that builds all the needed bits including fetching and building llvm/clang if necessary. BTW in our case, if necessary means if the llvm product .a file is absent since most of us don't change clang and it is annoying to have the llvm make figure out if it needs to do anything every time you change an lldb file...
Anyway, here's some options. Maybe there's more, this just what I came up with after a little brainstorming
1) Building the test suite on MacOSX requires an additional step beyond just building LLDB. You would have to build the test programs first, presumably the same way that you build LLVM assuming you use the CMake llvm build, then the test script would find the already-built test programs. Note that this is the same as what non MacOSX people would do, except that for us it wouldn't be an extra step, simply typing "ninja check-lldb" would automatically do everything, since building the test programs would be a dependency of running the test suite.
As Todd pointed out, there are some tests that rely on building and then hiding files, or building the tests in different ways, or stripping some of the output, etc. So I think the model where you build all the test programs up front isn't a great one for our test suite. If you wanted to use cmake, it would make more sense to have a pass that builds the Makefiles from the cmake whatever their input files are called, and then the actual test suite run would build the needed binaries using the Makefiles pretty much as it did before. Or if cmake can be used directly - like a Makefile - then we could switch the code in the testsuite to use them directly instead of invoking make. But anyway, "making the makefiles" can be done outside the test harness, but building the binaries needed for the tests still needs to be controlled by the test code directly.
2) Have parallel builds for the test programs on MacOSX and other platforms. MacOSX build would remain as-is, and other platforms would use CMake to build the tests. Maybe there could be an option in the test script to force MAKE-mode for backwards compatibility. This isn't that different from the current situation, since MacOSX and other platforms already have parallel build systems for LLDB itself. The biggest difference is that now when adding new tests, you would have to add both a CMake file and a Makefile. This is a small annoyance, but looking at the makefiles, they are all pretty simple and most of the logic is factored into the top-level Makefile. I think the same could be done for the CMake, so that new CMake files are a few lines.
This seems awkward.
3) Have MacOSX not use the native Xcode build anymore. This goes back to my original question though, about what the workflow is like. I imagine it's probably just historical, but why doesn't LLDB use the CMake build already? CMake can already generate Xcode projects, what are the differences between the Xcode projects generated by CMake versus the hand-maintained Xcode projects. Assuming they're significant enough to warrant serious concerns about the usability of a CMake-generated Xcode project, I wonder if it's something that's fixable to the point that using a CMake-generated Xcode project is almost the same as using the hand-maintained Xcode project. I've never used Xcode before, so I'm a bit clueless here.
We (or at least some of us) use Xcode because we like it. Dunno how easy it would be to get cmake to construct well-organized Xcode projects. Before we embark down this path - given that some of us really do use Xcode as our development platform - we'd have to have not just confidence not just that whatever difficulties there are in constructing useable projects could be overcome, but that there's somebody who uses Xcode who wants to maintain this bit. Having spent many years maintaining complex auto-conf based projects, I must admit I personally am very happy in my ignorance of cmake.