[Bug 25092] New: Test suit is flaky if 2 tests have the same file name

Bug ID 25092
Summary Test suit is flaky if 2 tests have the same file name
Product lldb
Version unspecified
Hardware PC
OS Linux
Status NEW
Severity normal
Priority P
Component All Bugs
Assignee lldb-dev@lists.llvm.org
Reporter tberghammer@google.com
CC llvm-bugs@lists.llvm.org
Classification Unclassified

If we have 2 test case with the same file name then the test suit will become
flaky for those tests.

In some ordering of the events (most likely when the 2 test with the same name
run at the same time) one of the test will fail with the following error:

Traceback (most recent call last):
  File
"/lldb-buildbot/lldbSlave/buildWorkingDir/llvm/tools/lldb/test/dotest.py", line
2019, in <module>
    resultclass=LLDBTestResult).run(suite)
  File
"/lldb-buildbot/lldbSlave/buildWorkingDir/llvm/tools/lldb/test/unittest2/runner.py",
line 162, in run
    test(result)
  File
"/lldb-buildbot/lldbSlave/buildWorkingDir/llvm/tools/lldb/test/unittest2/suite.py",
line 64, in __call__
    return self.run(*args, **kwds)
  File
"/lldb-buildbot/lldbSlave/buildWorkingDir/llvm/tools/lldb/test/unittest2/suite.py",
line 84, in run
    self._wrapped_run(result)
  File
"/lldb-buildbot/lldbSlave/buildWorkingDir/llvm/tools/lldb/test/unittest2/suite.py",
line 114, in _wrapped_run
    test._wrapped_run(result, debug)
  File
"/lldb-buildbot/lldbSlave/buildWorkingDir/llvm/tools/lldb/test/unittest2/suite.py",
line 116, in _wrapped_run
    test(result)
  File
"/lldb-buildbot/lldbSlave/buildWorkingDir/llvm/tools/lldb/test/unittest2/case.py",
line 417, in __call__
    return self.run(*args, **kwds)
  File
"/lldb-buildbot/lldbSlave/buildWorkingDir/llvm/tools/lldb/test/unittest2/case.py",
line 389, in run
    self.dumpSessionInfo()
  File
"/lldb-buildbot/lldbSlave/buildWorkingDir/llvm/tools/lldb/test/lldbtest.py",
line 1890, in dumpSessionInfo
    os.rename(src, dst)
OSError: [Errno 2] No such file or directory

We should handle the case when 2 test have the same file name or explicitly
disallow it with some check at test case loading time.

We should explicitly disallow it. You should be able to tell by looking at a test’s filename what it does. If two files have the same name, then you wonder why they aren’t the same test, and it leaves you with more questions than answers. If two tests have the same name and they actually should be different tests, then that’s a very good sign that one or both of them don’t have specific enough names.

labath@google.com changed bug 25092

What | Removed | Added |

  • | - | - |
    Status | NEW | RESOLVED |
    CC | | labath@google.com |
    Resolution | — | FIXED |

Comment # 4 on bug 25092 from labath@google.com

As of r249837, it should not be possible to have two tests with the same name,
so I am going to close this. Feel free to reopen if you think more needs to be
done about this.