The pre-built Windows LLDB binary has a dependency on an external python36.dll?

I just installed the pre-built LLVM9 binaries on a Windows machine and I noticed that LLDB.exe imports from python36.dll. Was this an intentional change from LLVM8? (which doesn’t depend on external python DLLs)

Trying to use the LLDB that comes with LLVM9, you’d get a pop-up complaining that python36.dll was not found (unless you happen to have on in your PATH) and LLDB fails to start.

Thanks,
Leonard.

There has been a lot of churn in the build process for Python on Windows over the past couple months.

Older versions included a pre-built Python DLL on Windows because of ABI compatibility. That issue is resolved, though, and I thought that was already over by version 7 or earlier.

Because other compatibility issues, it was decided that the Windows version could depend on 3.6+ while non-Windows versions might still rely on slightly older versions of Python.

That said, I didn’t expect an explicit dependency on python36.dll. Please file a bug. I’ll look through the revision history and see if I can loop in the folks who’ve been making changes in this area. I suspect it was an unintended side effect.

Here are some possibly related reviews. Note that some of these were abandoned, but I’m including them because the comments might give some context.

https://reviews.llvm.org/D69684
https://reviews.llvm.org/D69931
https://reviews.llvm.org/D67942
https://reviews.llvm.org/D69535
https://reviews.llvm.org/D68613
https://reviews.llvm.org/D68442

https://reviews.llvm.org/D68613

What kind of behavior did you expect?

pl

What kind of behavior did you expect?

I could be wrong, but I thought that previous versions of LLDB would use LoadLibrary() instead of linking to the import library?

Yes, that sounds plausible, but I don’t recall for sure. I think there’s a build-time option to say you don’t want Python at all, but I can’t remember if there was a load-as-needed option.

In any event, the current situation is what it is. What’s feasible and worth doing for the future?

  • Hard dependency (as we have right now)
  • Dynamically load Python DLL on startup if it exists, or provide a better error message with instructions
  • Dynamically load Python DLL on startup if it exists, otherwise disable Python-dependent features
  • Dynamically load a specific version of the Python DLL if/when needed
  • Dynamically load any supported Python DLL if/when needed

Yes, that sounds plausible, but I don't recall for sure. I think there's a build-time option to say you don't want Python at all, but I can't remember if there was a load-as-needed option.

I'm pretty sure we have never had explicit support for anything like this. The only way I can see this happening is if this fell out "accidentally" out of our lazy python initialization and some windows dll behavior, but I don't think windows has anything like that. (At least on linux, I know that lazy binding can delay library mismatch errors until the first time you call some function, but they won't help you if the library is not there at all.)

I think the more likely scenario is that python was disabled in the previous "official" releases, and that some of the python changes enabled it.

In any event, the current situation is what it is. What's feasible and worth doing for the future?

* Hard dependency (as we have right now)

I'm fine with that. We could add a note on the website that one needs to have python installed for this to work. Or we could disable python for the official releases.

* Dynamically load Python DLL on startup if it exists, or provide a better error message with instructions
* Dynamically load Python DLL on startup if it exists, otherwise disable Python-dependent features
* Dynamically load a specific version of the Python DLL if/when needed

All of these seem fine too, if anyone is willing to invest the time to make it work (it shouldn't be _that_ hard). Since python is pretty compartmentalized nowadays, it shouldn't relatively easy to disable python features at runtime instead of just exiting.

The main question I have here is should we dlopen python.dll, or some lldb wrapper of it (the entire "script interpreter" plugin).

I'd also like to note that this isn't the only external dependency of lldb. (Well, it might be on windows..) Lldb can use libcurses, libedit, libz, etc. Libedit is fairly likely to not be present on a random linux system. libcurses are almost certainly there, but it's not always a compatible version, etc.

* Dynamically load any supported Python DLL if/when needed

That might be tricky since the different versions are not binary compatible in general. But it is possible, as Apple folks have shown, though it amounts to building multiple copies of ScriptInterpreterPython and then choosing the right one at runtime.

pl

> * Dynamically load any supported Python DLL if/when needed
That might be tricky since the different versions are not binary compatible in
general. But it is possible, as Apple folks have shown, though it amounts to
building multiple copies of ScriptInterpreterPython and then choosing the
right one at runtime.

It's not just the python dll; it's the modules directory as well. My experiments with different versions of Python on Linux led me to just ship the right python with our distribution.

I saw things like building with 2.7.6 but using the 2.7.3 library/modules (and vice versa) would crash, and building with 2.7.6 but running with 2.6.x seemed to be OK, mostly. On Windows, I had crashes when loading Python 2.7.8 from python.org (built with VS 2008) in lldb built with VS 2013, so you have to think about other library dependencies too.

My conclusion - you MUST use the same python that lldb was built with. Period.

Yes, I think it’s pretty reasonable to expect a specific version of Python, especially if the pre-built Python DLLs for Windows are still built with versions as old as VS 2013. Once you get to VS 2015 or 2017, I think the compatibility improves.

Perhaps the best thing for the pre-built LLDB is to include the right Python DLL in the distro, assuming the licensing allows that.

The more sophisticated solutions are probably more work than is justified by the value.

FWIW, Python provides stable ABI for a subset of their API.

I’ve actually managed to create a version of LLDB that is Python-optional and Python version-agnostic (for versions 3.3 up). Though given the number of hoops I had to jump through to get there, I cannot recommend this approach for mainline LLDB.