Hi
Miniz (https://code.google.com/p/miniz/ ) is very small and performant implementation of zlib compression with api compatibility and it is public domain…
Miniz can be integrated directly into the llvm source code.
So it could be a good replacement or alternative to avoid zlib dependency…
If someone is interested i can provide a patch.
Regards
Christophe
What is the downside of Zlib dependency? I’m curious! 
-Filip
It might not be available, so all codepaths have to recover from zlib unavailability. For example, I don’t think compressed DWARF works on Windows.
If you build with configure+make, just run configure with —disable-zlib. We do this routinely, so I’m pretty sure it works.
If you build with configure+make, just run configure with —disable-zlib.
We do this routinely, so I’m pretty sure it works.
It "works" as in it builds. However, the support for it is disabled since
zlib::compress will always return StatusUnsupported, so the debug
information will just not be compressed. Effectively, compressed DWARF is
unsupported on Windows.
Saleem Abdulrasool <compnerd@compnerd.org> writes:
Sure, but if there aren't instructions for how to do it easily, then it's
effectively unsupported. There isn't really a canonical way to "install"
headers and libraries on Windows like you would on Linux.
It probably works on MinGW, but then you're a MinGW binary in a MinGW world.
Large software libraries like OpenCV (under 3rdparty directory) do include copies of zlib and friends and build it, for that reasons. The source code is just half a megabyte and I think the license is compatible. We could do the same with zlib or miniz.
obnoxious issues around. It creates all kind of fun whenever a security
issue is found or you have to fix the same portability issue in 100
different copies (*cough* gtest *cough*)
Joerg
Yes, this is incredibly annoying, so please avoid that. However I noticed that the current solution using CMake is broken, as I cannot enter my own ZLIB_ROOT, since no proper find_package is used.
IIUC zlib availability is tested and the library used if present. Do you
mean that LLVM does not use zlib on Windows when the library is present?Sure, but if there aren't instructions for how to do it easily, then it's
effectively unsupported. There isn't really a canonical way to "install"
headers and libraries on Windows like you would on Linux.It probably works on MinGW, but then you're a MinGW binary in a MinGW world.
I really don't like the idea of bundling zlib. I'm not aware of LLVM
bundling any external libraries currently and I don't think now is a
good time to start. Based on what has been discussed so far it sounds
like it's "possible" to build zlib (or use a prebuilt binary) on
windows and use it within LLVM so surely the right solution is...
* Document how zlib can by LLVM on windows.
* Improve our detection of zlib. I glanced at the the CMake files and
we aren't using ``find_package(zlib)`` which surprises me. Packages
exist in CMake to abstract away the issue of finding where header
files and libraries actually live. I glanced at the implementation of
FindZLIB.cmake and it looks like it has Windows support because I see
# Normal search.
set(_ZLIB_SEARCH_NORMAL
PATHS "[HKEY_LOCAL_MACHINE\\SOFTWARE\\GnuWin32\\Zlib;InstallPath]"
"$ENV{PROGRAMFILES}/zlib"
)
Thanks,
Dan.
I just had a go at hacking this so that we use find_package(ZLIB) if
LLVM_ENABLE_ZLIB is used. Does the attached patch work correctly?
If so I could send to llvm-commits for review.
use_find_package_zlib.patch (1.51 KB)
Reid Kleckner <rnk@google.com> writes:
IIUC zlib availability is tested and the library used if present. Do you
mean that LLVM does not use zlib on Windows when the library is present?Sure, but if there aren't instructions for how to do it easily, then it's
effectively unsupported. There isn't really a canonical way to "install"
headers and libraries on Windows like you would on Linux.
That's a non-issue. Downloading, compiling and installing zlib on
Windows takes about 10 minutes, for both MinGW and MSVC. There are
pre-compiled packages around too.
It is a good thing that LLVM works when zlib is not present, but
assuming that LLVM will not support certain zlib-related features
"because it is Windows" just shows how much misinformation some people
have about that OS.
It probably works on MinGW, but then you're a MinGW binary in a MinGW
world.
I think you are confusing MinGW with Cygwin. There is nothing special
about MinGW binaries. Moreover, for C code the libraries are compatible.
I've been using the same zlib binary on both MinGW(-w64) and MSVC for a
decade, across multiple toolset releases.
The "MinGW world" vs the "VS world" is restricted to C++, where
different ABIs are used by their respective compilers. That's nothing
new on Windows, which doesn't has a "platform C++ ABI" (nor Linux has,
because C++ has no special role on neither OS.) C++ compilers with
incompatible C++ ABIs are common on Windows. Actually, a MSVC++ version
is often incompatible with other versions.
It's half the way there. Configuring and compiling works, but linking fails, probably some definition mismatches...
But it can definitely be simplified, as due to the "REQUIRED" flag CMake will already fail if zlib isn't found, so no need for the
if (ZLIB_FOUND)
...
else()
message(FATAL_ERROR "zlib was requested but it could not be found")
endif()
It's half the way there. Configuring and compiling works, but linking fails, probably some definition mismatches...
It configured, compiled and linked okay for me. Could you look into it?
But it can definitely be simplified, as due to the "REQUIRED" flag CMake will already fail if zlib isn't found, so no need for the
if (ZLIB_FOUND)
...
else()
message(FATAL_ERROR "zlib was requested but it could not be found")
endif()
Thanks. I'm aware that's what REQUIRED would do but I thought I'd try
to be "defensive" just in case someone later on thinks removing
"REQUIRED" is a good idea...
If you think its unnecessary clutter I'd happily remove it.
I'll have another look tomorrow.
IMO, it is unnecessary clutter. It will fail if ${ZLIB_LIBRARIES} or ${ZLIB_INCLUDE_DIRS} are used anyways as they will be set to NOTFOUND. Or we go back to a silent-fail solution:
find_package(ZLIB QUIET)
if (LLVM_ENABLE_ZLIB AND ZLIB_FOUND)
set(HAVE_ZLIB_H 1)
set(HAVE_LIBZ 1)
list(APPEND CMAKE_REQUIRED_LIBRARIES ${ZLIB_LIBRARIES})
list(APPEND CMAKE_REQUIRED_INCLUDES ${ZLIB_INCLUDE_DIRS})
else()
set(HAVE_ZLIB_H 0)
set(HAVE_LIBZ 0)
endif()
I'll have another look tomorrow.
Thanks.
IMO, it is unnecessary clutter. It will fail if ${ZLIB_LIBRARIES} or ${ZLIB_INCLUDE_DIRS} are used anyways as they will be set to NOTFOUND. Or we go back to a silent-fail solution:
Fair enough. I've never been a big fan of silent failure. So perhaps
just this...
if (LLVM_ENABLE_ZLIB )
find_package(ZLIB REQUIRED)
set(HAVE_ZLIB_H 1)
set(HAVE_LIBZ 1)
list(APPEND CMAKE_REQUIRED_LIBRARIES ${ZLIB_LIBRARIES})
list(APPEND CMAKE_REQUIRED_INCLUDES ${ZLIB_INCLUDE_DIRS})
else()
set(HAVE_ZLIB_H 0)
set(HAVE_LIBZ 0)
endif()
Well, I just had a quick look again and there were two issues:
You added ${ZLIB_LIBRARIES} to CMAKE_REQUIRED_LIBRARIES not ${ZLIB_LIBRARY}
That was covered up under Linux/MacOSX by
if ( LLVM_ENABLE_ZLIB AND HAVE_LIBZ )
set(system_libs ${system_libs} z)
endif()
in Support's CMakeLists.txt
I corrected the first point and removed the second. Now, HAVE_LIBZ and HAVE_ZLIB_H aren't used at all anymore so I removed those as well.
Patch attached.
PS: I wouldn't really consider CMAKE_REQUIRED_LIBRARIES/CMAKE_REQUIRED_INCLUDES a clean solution. target_include_directories and target_link_libraries offer much finer control.
zlibfix.patch (1.88 KB)
Reid Kleckner <rnk@google.com> writes:
>
>> IIUC zlib availability is tested and the library used if present. Do you
>> mean that LLVM does not use zlib on Windows when the library is present?
>
>
> Sure, but if there aren't instructions for how to do it easily, then it's
> effectively unsupported. There isn't really a canonical way to "install"
> headers and libraries on Windows like you would on Linux.That's a non-issue. Downloading, compiling and installing zlib on
Windows takes about 10 minutes, for both MinGW and MSVC. There are
pre-compiled packages around too.It is a good thing that LLVM works when zlib is not present, but
assuming that LLVM will not support certain zlib-related features
"because it is Windows" just shows how much misinformation some people
have about that OS.
I haven't heard of anyone doing this successfully, and there isn't any
documentation, which is just a hair shy of being unsupported. I agree,
though, we can fix this. If GnuWin32 provides a usable zlib, that's great,
we should document how to use it.
It probably works on MinGW, but then you're a MinGW binary in a MinGW
> world.I think you are confusing MinGW with Cygwin. There is nothing special
about MinGW binaries. Moreover, for C code the libraries are compatible.
I've been using the same zlib binary on both MinGW(-w64) and MSVC for a
decade, across multiple toolset releases.
I did mean MinGW, I was really just referring to the C++ ABI, alternative
SDK headers, and CRT adapters.
The "MinGW world" vs the "VS world" is restricted to C++, where
different ABIs are used by their respective compilers. That's nothing
new on Windows, which doesn't has a "platform C++ ABI" (nor Linux has,
because C++ has no special role on neither OS.) C++ compilers with
incompatible C++ ABIs are common on Windows. Actually, a MSVC++ version
is often incompatible with other versions.
There are proposals to define such a platform C++ ABI for Windows, N4028:
https://isocpp.org/blog/2014/05/n4028
It's not clear to me that the ISO C++ standard is the right place to
declare that, but it suggests that Microsoft (or at least Herb) intend to
document the Visual C++ ABI.