Current state of the unit tests

Hi,

At the moment, I'm seeing two issues with the unit tests on my machine.

First, TestBase.LaunchModePreservesEnvironment is failing:

[ RUN ] TestBase.LaunchModePreservesEnvironment
/Users/vsk/src/llvm.org-lldbsan/llvm/tools/lldb/unittests/tools/lldb-server/tests/LLGSTest.cpp:30: Failure
Value of: llvm::detail::TakeExpected(Client.GetLatestStopReplyAs<StopReplyExit>())
Expected: succeeded with value (is an object whose given property is equal to 2-byte object <00-00>)
  Actual: succeeded with value 16-byte object <10-60 A7-04 01-00 00-00 01-00 00-00 00-00 00-30>, whose given property is 2-byte object <00-01>(is an object whose given property isn't equal to 2-byte object <00-00>)
[ FAILED ] TestBase.LaunchModePreservesEnvironment (67 ms)

I filed https://bugs.llvm.org/show_bug.cgi?id=36494 to track the issue.

Second, TestClient::SendMessage is generating quite a lot of "INFO" output which clutters up the terminal. Pavel, would you mind if I removed this logging?

thanks,
vedant

Yeah, we should probably do that. The idea here was that the packet
log would provide you with additional context for the situation when
the test fails, but it *is* very verbose. I'll have to come up with a
better solution for error reporting here.

Cool, I’ll work up a patch for this.

And thanks for commenting on PR36494, I’m testing a fix out right now :).

vedant

As of this afternoon, it seems ninja check-lldb runs only the lit tests and not the dotest.py tests. Was this an intentional change?

check-lldb-lit should just be a dependency of check-lldb, so the dotest.py tests should still run.

Are one of the lit tests failing? That might explain why subsequent tests aren’t run.

vedant

Actually, it appears one of the lit tests is unexpectedly passing:

Unexpected Passing Tests (1):
lldb :: Expr/TestCallStdStringFunction.test

lit then returns an error code, and ninja bails before starting the dotest.py tests:

FAILED: cmd.exe /C “cd /D D:\src\llvm\build\mono\tools\lldb\lit && C:\python_35\python_d.exe D:/src/llvm/build/mono/./bin/llvm-lit.py -sv --param lldb_site_config=D:/src/llvm/build/mono/tools/lldb/lit/lit.site.cfg --param lldb_unit_site_config=D:/src/llvm/build/mono/tools/lldb/lit/Unit/lit.site.cfg D:/src/llvm/build/mono/tools/lldb/lit”
ninja: build stopped: subcommand failed.

I’m also seeing windows appear and quickly vanish a several times while running the lit tests.

Yeah, if a lit test fails, the dotest tests will not get run. That is
fine, but having a target which only runs dotest tests would probably
be nice as well.

That's because the tests run inferiors and lldb on windows will always
run them in a separate console window. IIRC, there is a special hack
in dotest, which prevents opening windows for testing. You probably
need something like that for lit tests as well.

Hi,

At the moment, I'm seeing two issues with the unit tests on my machine.

First, TestBase.LaunchModePreservesEnvironment is failing:

[ RUN ] TestBase.LaunchModePreservesEnvironment
/Users/vsk/src/llvm.org-lldbsan/llvm/tools/lldb/unittests/tools/lldb-server/tests/LLGSTest.cpp:30: Failure
Value of: llvm::detail::TakeExpected(Client.GetLatestStopReplyAs<StopReplyExit>())
Expected: succeeded with value (is an object whose given property is equal to 2-byte object <00-00>)
Actual: succeeded with value 16-byte object <10-60 A7-04 01-00 00-00 01-00 00-00 00-00 00-30>, whose given property is 2-byte object <00-01>(is an object whose given property isn't equal to 2-byte object <00-00>)
[ FAILED ] TestBase.LaunchModePreservesEnvironment (67 ms)

I filed https://bugs.llvm.org/show_bug.cgi?id=36494 to track the issue.

^ This issue is resolved now with a cmake fix.

Second, TestClient::SendMessage is generating quite a lot of "INFO" output which clutters up the terminal. Pavel, would you mind if I removed this logging?

^ I'll let Pavel decide the best way to deal with this one.

vedant