Problem: cmake-built libzmq is not binary-compatible with autotools builds

[minrk]

tl;dr: if a library links zeromq built normally and the same version of ibzmq is rebuilt with cmake, everything that had linked to libzmq will be broken until they are rebuilt.

Versioning libraries is fraught and complicated on different platforms. #2886 fixes SONAME of cmake files, but does not fix the library versioning. After #2886, filenames are now mostly correct (on linux, not quite on mac). This does not, however, fix the versions of the library itself. See this output for cmake and autotools builds of e2ed5a3:

minrk[11:39]~/dev/zmq/libzmq (master) $ otool -L /tmp/normal/lib/libzmq.dylib
/tmp/normal/lib/libzmq.dylib:
	/tmp/normal/lib/libzmq.5.dylib (compatibility version 7.0.0, current version 7.4.0)
minrk[11:48]~/dev/zmq/libzmq (master) $ otool -L /tmp/cmake/lib/libzmq.dylib
/tmp/cmake/lib/libzmq.dylib:
	libzmq.5.dylib (compatibility version 5.0.0, current version 5.1.4)

Now, what CMake does isn't necessarily wrong here. Ultimately, all that matters is that builds are internally consistent. The problem comes from having two build systems, and those build systems having incompatible idiosyncrasies.

In order for cmake and autotools builds to be compatible, The compatibility version and current version for cmake must be exactly identical for both builds. The reasons for libtool's version choices are a bit weird and historical, but they are necessary for cmake-built libzmq to be compatible with normal libzmq.

Right now, we have LTVER=6.4.1. That means:

• current is 6
• implementation is 4
• age is 1
• ABI version is current - age (6-1=5)
• soversion is ABI (5)
• compatibility version is current + 1 (6+1=7)
• current version is compatibility.revision.0 (7.4.0)

CMake has to match all of libtool's platform-specific weirdness if it wants to be compatible with normal builds of libzmq. I'm not sure how to do that.

I think having two build systems that are incompatible is a source of major problems. My recommendation would be to deprecate one or the other, or at least place big caveats that anything that adopts one must never switch without ensuring that all downstream packages are rebuilt. Personally, I would recommend that CMake builds of zeromq be reserved for Windows.

cc @SylvainCorlay

[SylvainCorlay]

Thanks for opening this @minrk. As per our discussions in conda-forge/zeromq-feedstock#25 I agree that the parallel existence of multiple build systems giving slightly different results is the main source of the troubles that we had in the context of packaging for conda.

On whether libzmq should ultimately be built with autotools or cmake, I think that we should use cmake.

Here are the main reasons for preferring cmake over autotools

• cmake dominates in terms of adoption in the wider C++ community and has become the de facto standard. Even boost's steering council recently announced that they were dropping their build system for cmake http://boost.2283326.n4.nabble.com/CMake-Announcement-from-Boost-Steering-Committee-tt4696934.html
• The cmake still has to be maintained for the windows builds, so the maintenance burden would not be much reduced if it was to not be recommended on unix platform.
• Even within the zeromq organization, many subprojects actually use cmake: cppzmq, azmq and others.

Finally, there are technical arguments to it. In the modern CMake design were dependencies are expressed through targets, a library author can express a lot about how the library should be used (compilation and linker flags, include directories to consider, transitivity of dependencies and much more) depending on how one uses the library. In scenarios where third-party developers write their own FindModule scripts, they have to make a lot of guessing about what the actual interface requirements are. The current trend is that if you need to write a findModule for a third-party library, report this as a bug to the authors.

I have also been frustrated at times with cmake, but I think that it would have been the case for any build system.

[bluca]

On whether libzmq should ultimately be built with autotools or cmake, I think that we should use cmake.

You are of course very welcome to maintain support for CMake and use it, and I'll keep doing my best to help as I'm sure will others. But sorry, I am also going to keep maintaining and using autotools, and it's still going to be used for DEB/RPM (one day Arch too, maybe) packaging and by all the distros. For OSX, I don't maintain the Brew recipe but I suspect they are not going to change either.

Personally, I would recommend that CMake builds of zeromq be reserved for Windows.

I would agree, and would recommend to use autotools on *nix as well, but that's only my personal choice - I don't think we should start to force users at this point - it would be a backward-incompatible change, and everybody just hates those, especially for libraries :-)

[SylvainCorlay]

[...] but does not fix the library versioning [...]
In order for cmake and autotools builds to be compatible, The compatibility version and current version for cmake must be exactly identical for both builds. The reasons for libtool's version choices are a bit weird and historical, but they are necessary for cmake-built libzmq to be compatible with normal libzmq.

Right now, we have LTVER=6.4.1. That means:

current is 6
implementation is 4
age is 1
ABI version is current - age (6-1=5)
soversion is ABI (5)
compatibility version is current + 1 (6+1=7)
current version is compatibility.revision.0 (7.4.0)
CMake has to match all of libtool's platform-specific weirdness if it wants to be compatible with normal builds of libzmq. I'm not sure how to do that.

is this only an OS X issue?

[minrk]

is this only an OS X issue?

Possibly, I'm not sure.

[bluca]

if the SONAME is wrong then it affects any *nix

[minrk]

SONAME is indeed fixed by #2886. However, darwin, at least, has a more sophisticated version/compatibility system than just filenames, though.

[SylvainCorlay]

Note, the following patch to CMakeLists makes the file names generated by autotools and cmake be the same, however, the compatibility version and current version showed by otool are still inconsistent.

---
 CMakeLists.txt | 26 ++++++++++++++++++--------
 1 file changed, 18 insertions(+), 8 deletions(-)

diff --git a/CMakeLists.txt b/CMakeLists.txt
index bbc451e..7421827 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -792,14 +792,24 @@ else ()
 
   if (BUILD_SHARED)
     add_library (libzmq SHARED $<TARGET_OBJECTS:objects> ${public_headers} ${html-docs} ${readme-docs} ${zmq-pkgconfig} ${CMAKE_CURRENT_BINARY_DIR}/version.rc)
-    # NOTE: the SOVERSION and VERSION MUST be the same as the one generated by libtool! It is NOT the same as the version of the package.
-    set_target_properties (libzmq PROPERTIES
-                          COMPILE_DEFINITIONS "DLL_EXPORT"
-                          PUBLIC_HEADER "${public_headers}"
-                          VERSION "5.1.4"
-                          SOVERSION "5"
-                          OUTPUT_NAME "zmq"
-                          PREFIX "lib")
+    # SOVERSION and VERSION are set so as to align with the behavior of libtool.
+    if (APPLE)
+        set_target_properties (libzmq PROPERTIES
+                               COMPILE_DEFINITIONS "DLL_EXPORT"
+                               PUBLIC_HEADER "${public_headers}"
+                               VERSION "5"
+                               SOVERSION "5"
+                               OUTPUT_NAME "zmq"
+                               PREFIX "lib")
+    else (APPLE)
+        set_target_properties (libzmq PROPERTIES
+                               COMPILE_DEFINITIONS "DLL_EXPORT"
+                               PUBLIC_HEADER "${public_headers}"
+                               VERSION "5.1.4"
+                               SOVERSION "5"
+                               OUTPUT_NAME "zmq"
+                               PREFIX "lib")
+    endif (APPLE)
     if (ZMQ_BUILD_FRAMEWORK)
       set_target_properties (libzmq PROPERTIES
                             FRAMEWORK TRUE
-- 
2.5.3

[minrk]

@SylvainCorlay thanks! I think the compatibility_version is not fixable as long as cmake unconditionally uses SOVERSION for the compatibility version. If you could override compatibility/current version by hand, then I think the compatibility issues would be fixed.

[SylvainCorlay]

Indeed, I opened an issue upstream about this.

https://gitlab.kitware.com/cmake/cmake/issues/17652

There might be a workaround.

[stale]

This issue has been automatically marked as stale because it has not had activity for 365 days. It will be closed if no further activity occurs within 56 days. Thank you for your contributions.

[SylvainCorlay]

I don't think that this can be closed. This is a legitimate issue.

[SylvainCorlay]

@minrk @bluca FYI Kitware just merged a PR into CMake that should allow to fix this issue.

https://gitlab.kitware.com/cmake/cmake/merge_requests/4274

[Schamschula]

I just ran into this issue while trying to convert the zmq Portfile under MacPorts to use cmake rather than autotools.

MacPorts has been using autotools, but I am building a project that requires ZeroMQConfig.cmake and ZeroMQConfigVersion.cmake to be installed.

It's a shame that all the dependent ports would have to be rebuilt, as both the compatibility version and current version are now lower.

[Durant35]

Seems so strange... For tag v4.3.0, the generated library is libzmq.so.5.2.0 since https://github.com/zeromq/libzmq/blob/v4.3.0/CMakeLists.txt#L1077, and all documents are followings the first version number: http://api.zeromq.org/

[bluca]

Source version != ABI version

[stale]

This issue has been automatically marked as stale because it has not had activity for 365 days. It will be closed if no further activity occurs within 56 days. Thank you for your contributions.