Bug ID 25562
Summary Some failing tests are being reported as passes
Product lldb
Version unspecified
Hardware PC
OS Windows NT
Status NEW
Severity normal
Priority P
Component All Bugs
Assignee lldb-dev@lists.llvm.org
Reporter zturner@google.com
CC llvm-bugs@lists.llvm.org
Classification Unclassified
If I run this test manually, two of the tests fails. Example:
d:\src\llvmbuild\ninja\lldb-test-traces>c:\python27_lldb\x86\python_d.exe
D:\src\llvm\tools\lldb\test\dotest.py -q --arch=i686 --executable
D:/src/llvmbuild/ninja/bin/lldb.exe -s D:/src/llvmbuild/ninja/lldb-test-traces
-u CXXFLAGS -u CFLAGS --enable-crash-dialog -C
d:\src\llvmbuild\ninja_release\bin\clang.exe
D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test\functionalities\process_launch
--no-multiprocess
The 'lldb-mi' executable cannot be located. The lldb-mi tests can not be run
as a result.
UNSUPPORTED: LLDB (d:\src\llvmbuild\ninja_release\bin\clang.exe-i686) ::
test_environment_with_special_char_dsym
(TestProcessLaunch.ProcessLaunchTestCase) (dsym tests)
FAIL: LLDB (d:\src\llvmbuild\ninja_release\bin\clang.exe-i686) ::
test_environment_with_special_char_dwarf
(TestProcessLaunch.ProcessLaunchTestCase)
FAIL: LLDB (d:\src\llvmbuild\ninja_release\bin\clang.exe-i686) ::
test_environment_with_special_char_dwo
(TestProcessLaunch.ProcessLaunchTestCase)
UNSUPPORTED: LLDB (d:\src\llvmbuild\ninja_release\bin\clang.exe-i686) ::
test_io_dsym (TestProcessLaunch.ProcessLaunchTestCase) (dsym tests)
PASS: LLDB (d:\src\llvmbuild\ninja_release\bin\clang.exe-i686) :: test_io_dwarf
(TestProcessLaunch.ProcessLaunchTestCase)
PASS: LLDB (d:\src\llvmbuild\ninja_release\bin\clang.exe-i686) :: test_io_dwo
(TestProcessLaunch.ProcessLaunchTestCase)
UNSUPPORTED: LLDB (d:\src\llvmbuild\ninja_release\bin\clang.exe-i686) ::
test_set_working_dir_dsym (TestProcessLaunch.ProcessLaunchTestCase) (dsym
tests)
When I run the multiprocess test runner via dosep, it's being reported as a
success.
410 out of 410 test suites processed - TestAddDsymCommand.py
Ran 410 test suites (1 failed) (0.243902%)
Ran 318 test cases (4 failed) (1.257862%)
Failing Tests (1)
FAIL: LLDB (suite) :: TestBreakpointLanguage.py (Windows zturner-win81 8
6.2.9200 AMD64 Intel64 Family 6 Model 45 Stepping 7, GenuineIntel)
[79964 refs]
ninja: build stopped: subcommand failed.
Stepping through the code, in dosep.py when it goes to print the summary list
of failed tests, 'TestProcessLaunch.py' is in the list of passed tests. So
something is wrong here, and a potentially unknown number of tests are
currently being misreported.