Python packaging change

Subprojects LLDB
#1

I recently updated by tree and had problems running lldb due to Python issues. Below is what I've been able to determine is the issue. I thought asking the list first for directional advice would make more sense prior to suggesting a patch.

This revision changed the Python files into a Python package:
http://llvm.org/viewvc/llvm-project?view=rev&revision=155514

However, this broke running on FreeBSD since the finish-swig-Python-LLDB.sh script is only run from an Xcode build even though the script appears to support both LLDB.framework and non-Darwin builds. For non-Darwin builds the Python file install is done in source/Interpreter/Makefile (which was not changed to support the new packaging and usage).

There are probably 3 locations for installing the Python packages:

1. Framework installs for Darwin
2. In-tree installs for testing lldb local to the source tree
3. Installed lldb for general use.

I believe the Python packaging should be consistent across these install locations even though the actual location may vary. The question is, where should the packaging be done for these to be kept consistent? Should non-Darwin systems use the shell script instead of the Makefile? Or should both be kept in sync?

Also, two other related issues:
1. The finish-swig-Python-LLDB.sh shell script introduced a sh incompatibility by using a bash function.
2. It is unclear that the non-Darwin need or want some of the Cocoa/objc python files which would mean changing the packaging and the Python imports used in ScriptInterpreterPython.cpp to optionally install those files.

Thanks,
Mark

#2

I recently updated by tree and had problems running lldb due to Python issues.
Below is what I've been able to determine is the issue. I thought asking the
list first for directional advice would make more sense prior to suggesting a
patch.

This revision changed the Python files into a Python package:
http://llvm.org/viewvc/llvm-project?view=rev&revision=155514

However, this broke running on FreeBSD since the finish-swig-Python-LLDB.sh
script is only run from an Xcode build even though the script appears to
support both LLDB.framework and non-Darwin builds. For non-Darwin builds the
Python file install is done in source/Interpreter/Makefile (which was not
changed to support the new packaging and usage).

And now it is. Try this patch.

There are probably 3 locations for installing the Python packages:

1. Framework installs for Darwin
2. In-tree installs for testing lldb local to the source tree
3. Installed lldb for general use.

I believe the Python packaging should be consistent across these install
locations even though the actual location may vary. The question is, where
should the packaging be done for these to be kept consistent? Should
non-Darwin systems use the shell script instead of the Makefile? Or should
both be kept in sync?

The problem with the shell script is that, when run on Darwin, it assumes it was run from Xcode. Not all of us use Xcode to build on Darwin ;). Personally, I would like to get rid of the duplication of effort between the shell scripts and the makefiles, but I haven't figured out how I want to do that yet.

2. It is unclear that the non-Darwin need or want some of the Cocoa/objc
python files which would mean changing the packaging and the Python imports
used in ScriptInterpreterPython.cpp to optionally install those files.

Imagine that you're debugging a program on a remote Mac OS X system.

Chip

python-packaging.patch (8.85 KB)

#3

Thank you for the patch. For FreeBSD I had to switch a couple of the echos from using \c in the string to using "echo -n". After that it worked great for the in-tree version.

Are you going to submit the patch out to lldb-commit?

Thanks,
Mark

#4

I recently updated by tree and had problems running lldb due to Python issues.
Below is what I've been able to determine is the issue. I thought asking the
list first for directional advice would make more sense prior to suggesting a
patch.

This revision changed the Python files into a Python package:
http://llvm.org/viewvc/llvm-project?view=rev&revision=155514

However, this broke running on FreeBSD since the finish-swig-Python-LLDB.sh
script is only run from an Xcode build even though the script appears to
support both LLDB.framework and non-Darwin builds. For non-Darwin builds the
Python file install is done in source/Interpreter/Makefile (which was not
changed to support the new packaging and usage).

And now it is. Try this patch.

Thank you for the patch. For FreeBSD I had to switch a couple of the echos from using \c in the string to using "echo -n".

Yeah, I wanted to use "echo -n" but all that did on my computer was write '-n' to the files.

After that it worked great for the in-tree version.

That was unexpected, considering that I haven't modified the test makefiles to point to the lldb package yet.

Are you going to submit the patch out to lldb-commit?

Not yet. I need to figure out a portable way to write text to a file without a trailing newline.

Chip

#5

I recently updated by tree and had problems running lldb due to Python issues.
Below is what I've been able to determine is the issue. I thought asking the
list first for directional advice would make more sense prior to suggesting a
patch.

This revision changed the Python files into a Python package:
http://llvm.org/viewvc/llvm-project?view=rev&revision=155514

However, this broke running on FreeBSD since the finish-swig-Python-LLDB.sh
script is only run from an Xcode build even though the script appears to
support both LLDB.framework and non-Darwin builds. For non-Darwin builds the
Python file install is done in source/Interpreter/Makefile (which was not
changed to support the new packaging and usage).

And now it is. Try this patch.

Thank you for the patch. For FreeBSD I had to switch a couple of the echos from using \c in the string to using "echo -n".

Yeah, I wanted to use "echo -n" but all that did on my computer was write '-n' to the files.

What OS and OS version are you running this on? This is the patch to your patch that worked for me:

% diff -u python-packaging.patch.orig python-packaging.patch
--- python-packaging.patch.orig 2012-05-14 21:38:12.000000000 -0700
+++ python-packaging.patch 2012-05-14 13:45:51.000000000 -0700
@@ -74,9 +74,9 @@
  + $(foreach file,$(LLDB_PACKAGE_$(subpackage)_FILES), \
  + $(CP) "$(file)" "$(PYTHON_DIR)/$(LLDB_PACKAGE_$(subpackage))"; \
  + ) \
-+ echo "__all__ = [\c" >$$init_file; \
-+ echo "$(patsubst %,\"%\"$(comma),\
-+ $(basename $(notdir $(LLDB_PACKAGE_$(subpackage)_FILES))))\c" >>$$init_file; \
++ echo -n "__all__ = [" >$$init_file; \
++ echo -n "$(patsubst %,\"%\"$(comma),\
++ $(basename $(notdir $(LLDB_PACKAGE_$(subpackage)_FILES))))" >>$$init_file; \
  + echo "]" >>$$init_file; \
  + echo "for x in __all__:" >>$$init_file; \
  + echo " __import__('lldb.$(subst /,.,$(LLDB_PACKAGE_$(subpackage))).'+x)" >>$$init_file; \

In other words, replace these lines:

   echo "__all__ = [\c" >$$init_file; \
   echo "$(patsubst %,\"%\"$(comma),\
     $(basename $(notdir $(LLDB_PACKAGE_$(subpackage)_FILES))))\c"

with:

   echo -n "__all__ = [" >$$init_file; \
   echo -n "$(patsubst %,\"%\"$(comma),\
     $(basename $(notdir $(LLDB_PACKAGE_$(subpackage)_FILES))))"

After that it worked great for the in-tree version.

That was unexpected, considering that I haven't modified the test makefiles to point to the lldb package yet.

For my tests I manually set PYTHONPATH so it worked for me without complaining about import errors which the trunk version has been emitting.

Are you going to submit the patch out to lldb-commit?

Not yet. I need to figure out a portable way to write text to a file without a trailing newline.

Sounds good. Using echo -n should do the trick since that's been around for quite some time. Let me know the OS and I might be able to repro to help look at it.

Thanks,
Mark

#6

<snip>

Are you going to submit the patch out to lldb-commit?

Not yet. I need to figure out a portable way to write text to a file without a trailing newline.

Use printf instead. It's basically impossible to use echo portably for
this.

John Bytheway