Not sure exactly what it is, but all the tests are failing due to some bad assumptions of unicode vs. str on Python 2 vs. 3 if I had to guess.
I am not going to be able to look at details on that, but here’s a link to the log on the OS X builder:
http://lab.llvm.org:8080/green/job/lldb_build_test/16166/console
Not sure exactly what it is, but all the tests are failing due to some bad assumptions of unicode vs. str on Python 2 vs. 3 if I had to guess.
Author: zturner
New Revision: 258759
URL: http://llvm.org/viewvc/llvm-project?rev=258759&view=rev
Log:
Write the session log file in UTF-8.
Previously we were writing in the default encoding, which depends
on the operating system and is not guaranteed to be unicode aware.
On Python 3, this would lead to a situation where writing unicode
text to the log file generates an exception. The fix here is to
write session logs using the proper encoding, which incidentally
fixes another test, so xfail is removed from that.
sounds like a likely culprit from what you’re saying
I am not going to be able to look at details on that, but here’s a link to the log on the OS X builder:
Do you want me to revert?
http://lab.llvm.org:8080/green/job/lldb_build_test/16166/console
–
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
Thanks,
- Enrico
egranata@.com 
27683
Well our whole test suite just stopped running, so yes.
I think I see what happened w/r/t why no emails when out when the build went heavy red. (Well they went out internally, but not externally). When I made the change on Friday to improve the workflow for the Green Dragon OS X builder and test output, I switched email over to the builder step, which doesn’t know anything about who made which changes. So it didn’t know who to put on the blame list for the broken build. Drats, I’ll have to figure that out.
I’d really prefer to have all those stages happening in one build step to keep it clear what’s going on.
sorry, yea I stuck around for a while after that patch waiting for emails, but nothing came through. Please revert in the meantime, I’ll work on a fix tomorrow.
Oh no worries, I borked the builder email on our currently cumbersome multi-step process.
They’re failing with a permutation of this:
Traceback (most recent call last):
File "/Users/buildslave/jenkins/sharedspace/lldb@2/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 2224, in dsym_test_method
return attrvalue(self)
File "/Users/buildslave/jenkins/sharedspace/lldb@2/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 561, in wrapper
return func(self, *args, **kwargs)
File "/Users/buildslave/jenkins/sharedspace/lldb@2/lldb/packages/Python/lldbsuite/test/tools/lldb-server/inferior-crash/TestGdbRemoteAbort.py", line 31, in test_inferior_abort_received_debugserver
self.init_debugserver_test()
File "/Users/buildslave/jenkins/sharedspace/lldb@2/lldb/packages/Python/lldbsuite/test/tools/lldb-server/gdbremote_testcase.py", line 199, in init_debugserver_test
(self.named_pipe_path, self.named_pipe, self.named_pipe_fd) = self.create_named_pipe()
File "/Users/buildslave/jenkins/sharedspace/lldb@2/lldb/packages/Python/lldbsuite/test/tools/lldb-server/gdbremote_testcase.py", line 125, in create_named_pipe
self.addTearDownHook(shutdown_named_pipe)
File "/Users/buildslave/jenkins/sharedspace/lldb@2/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 1589, in addTearDownHook
print("Adding tearDown hook:", getsource_if_available(hook), file=sbuf)
File "/Users/buildslave/jenkins/sharedspace/lldb@2/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 265, in __exit__
print(self.getvalue(), file=self.session)
TypeError: must be unicode, not str
Config=x86_64-/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
Traceback (most recent call last):
File "/Users/buildslave/jenkins/sharedspace/lldb@2/lldb/test/dotest.py", line 7, in <module>
lldbsuite.test.run_suite()
File "/Users/buildslave/jenkins/sharedspace/lldb@2/lldb/packages/Python/lldbsuite/test/dotest.py", line 1089, in run_suite
resultclass=test_result.LLDBTestResult).run(configuration.suite)
File "/Users/buildslave/jenkins/sharedspace/lldb@2/lldb/third_party/Python/module/unittest2/unittest2/runner.py", line 162, in run
test(result)
File "/Users/buildslave/jenkins/sharedspace/lldb@2/lldb/third_party/Python/module/unittest2/unittest2/suite.py", line 65, in __call__
return self.run(*args, **kwds)
File "/Users/buildslave/jenkins/sharedspace/lldb@2/lldb/third_party/Python/module/unittest2/unittest2/suite.py", line 85, in run
self._wrapped_run(result)
File "/Users/buildslave/jenkins/sharedspace/lldb@2/lldb/third_party/Python/module/unittest2/unittest2/suite.py", line 115, in _wrapped_run
test._wrapped_run(result, debug)
File "/Users/buildslave/jenkins/sharedspace/lldb@2/lldb/third_party/Python/module/unittest2/unittest2/suite.py", line 117, in _wrapped_run
test(result)
File "/Users/buildslave/jenkins/sharedspace/lldb@2/lldb/third_party/Python/module/unittest2/unittest2/case.py", line 433, in __call__
return self.run(*args, **kwds)
File "/Users/buildslave/jenkins/sharedspace/lldb@2/lldb/third_party/Python/module/unittest2/unittest2/case.py", line 361, in run
success = self.runMethod(testMethod, result)
File "/Users/buildslave/jenkins/sharedspace/lldb@2/lldb/third_party/Python/module/unittest2/unittest2/case.py", line 413, in runMethod
result.addError(self, sys.exc_info())
File "/Users/buildslave/jenkins/sharedspace/lldb@2/lldb/packages/Python/lldbsuite/test/test_result.py", line 148, in addError
method()
File "/Users/buildslave/jenkins/sharedspace/lldb@2/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 1666, in markError
print("ERROR", file=sbuf)
File "/Users/buildslave/jenkins/sharedspace/lldb@2/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 265, in __exit__
print(self.getvalue(), file=self.session)
TypeError: must be unicode, not str
There's a pattern for using a call that guarantees contents are converted to either (say) Unicode or Bytes, where the right way to do it depends on Python 2 or Python 3. I see that this isn't causing a problem on the Linux builder, which is a bit puzzling unless they are using Python 3. (Or something else is going on that isn't obvious).
I think I may leave it as is and try to fix it. If I can't figure it out quickly-ish tonight, I will revert and we can figure out the right fix tomorrow. (i.e. I'm not going to revert it right away if I can fix it quickly).
-Todd
Should be reverted in 258791.
Hah the comedy of late night emails. We all go away for hours, then all get back here within minutes 
(Zachary - I will still have a look at it tonight since I am curious why we weren’t seeing it on all the (what I think are) Python 2.7-based systems).
I’m also not sure why Linux isn’t failing. Looking at the documentation for io.write object, i see this:
Write the unicode string s to the stream and return the number of characters written.
What’s the pattern you’re referring to? You can’t convert a string to a unicode without specifying an encoding, and it seems annoying to have to do that on every single call to print.
It’s in item 3 from Effective Python, by Brett Slatkin, which goes over having methods that always go to unicode or to byte streams taking either unicode or byte style strings, for both Python 2 and Python 3. Essentially you figure out what you want it to be in, and you write a couple helper routes to go in either the “to unicode” or the “to bytes” direction. It basically looks at the type of the string/bytes you give it, and makes sure it becomes what you need. It’s going to assume an encoding like utf-8.
Okay we’re back to green here:
http://lab.llvm.org:8080/green/job/lldb_build_test/16173/
Thanks, Enrico!
Zachary, I may let this rest until the morning. If you want to try something else, shoot me a patch and I’ll gladly try it.
-Todd
No worries, worst case scenario a sledgehammer solution is to change all the places where we write to the session file to convert to unicode first (which would be a trivial conversion, since everything is going to be ascii, which is already valid utf 8).
The reason a problem arose at all is because TestCxxWcharT.py tried to write actual unicode characters to the session file. I don’t think there’s any way to prevent that because the characters could appear in a backtrace, in a variable name, or in a test that is specifically testing unicode. So we can’t just have the one place that needs to write unicode encode it as bytes because there’s no one place.
I’ll sleep on it and try to see if there’s a better solution. Maybe we can just make a function called write_session_file() that takes either a string or a unicode, converts it to a unicode if it’s not already one, and then writes.
Can one of you guys try out this patch and see if it works? If so I’ll commit it.
I don’t know of a way to make this “elegant”. i.e. a single syntax / paradigm that works in both versions without introducing any helper functions.
diff.txt (3.27 KB)
Bump. Can I re-submit this?
Oh missed this. I’ll give it a shot, hang on.
I’m still getting a lot of these:
Traceback (most recent call last):
File “test/dotest.py”, line 7, in
lldbsuite.test.run_suite()
File “/Users/tfiala/src/lldb-tot/lldb/packages/Python/lldbsuite/test/dotest.py”, line 1089, in run_suite
resultclass=test_result.LLDBTestResult).run(configuration.suite)
File “/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/runner.py”, line 162, in run
test(result)
File “/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/suite.py”, line 65, in call
return self.run(*args, **kwds)
File “/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/suite.py”, line 85, in run
self._wrapped_run(result)
File “/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/suite.py”, line 115, in _wrapped_run
test._wrapped_run(result, debug)
File “/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/suite.py”, line 117, in _wrapped_run
test(result)
File “/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/case.py”, line 433, in call
return self.run(*args, **kwds)
File “/Users/tfiala/src/lldb-tot/lldb/third_party/Python/module/unittest2/unittest2/case.py”, line 369, in run
self.dumpSessionInfo()
File “/Users/tfiala/src/lldb-tot/lldb/packages/Python/lldbsuite/test/lldbtest.py”, line 1810, in dumpSessionInfo
print(u"Session info generated @", datetime.datetime.now().ctime(), file=self.session)
TypeError: must be unicode, not str
Humm… it’s a unicode literal, not sure why it’s complaining. I guess I’ll have to crack open my linux machine and see what’s going on tomorrow.
Okay. I’m stuck fixing the OS X build, but if you have something else to check, feel free to send it over my way and I can run it on another machine.