Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add component support to boost using boostdep #2097

Merged
merged 53 commits into from
Dec 12, 2020
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
59f72e3
boost: build dependency tree
madebr Jun 30, 2020
e7304b1
boost: add components using information of boostdep tool
madebr Jun 30, 2020
55e46cc
boost: only list buildable components in dependencies + add component…
madebr Jul 1, 2020
91eefc6
boost: fix -U option on fresh start + no buildables in yml
madebr Jul 1, 2020
532ffe4
boost: git submodule deinit + init to make sure new/older boost libra…
madebr Jul 1, 2020
09a2c80
boost: remove dep of graph on graph_parallel and remove dep of python…
madebr Jul 1, 2020
62860fa
boost: parse boost jam files to detect boost libraries
madebr Jul 1, 2020
d458d51
boost: fix dependency support + test_package of python
madebr Jul 21, 2020
c77292d
boost: add numpy test
madebr Jul 21, 2020
ab4415a
boost: fix builds with python disabled + fix boost<1.69
madebr Jul 21, 2020
d6fc24c
boost: use python_version as python version
madebr Jul 21, 2020
72d1437
boost: first test option, 2nd test dependencies, 3rd test libraries
madebr Jul 21, 2020
45501a8
boost: bump cmake minimum required version to 3.5 to required Boost::…
madebr Jul 21, 2020
481e840
boost: add numpy option + python version in options + test boost comp…
madebr Jul 21, 2020
df88336
boost: use Boost::boost + don't add numpy as option
madebr Jul 24, 2020
50e1188
boost: add extra find_package when boost is built header_only
madebr Aug 3, 2020
44cd287
boost: remove outdated comment in package_info
madebr Aug 3, 2020
3538e1c
boost: store dpeendencies in self.recipe_folder (=export folder)
madebr Aug 19, 2020
777c41c
boost: load configuration options from dependencies + add 1.74.0 yml
madebr Aug 19, 2020
698ea8c
boost: lowercase test_package + remove android workaround because we'…
madebr Aug 19, 2020
ba3aa40
boost: run find_package for each component separately
madebr Aug 19, 2020
30c9b7a
Merge remote-tracking branch 'origin/master' into boost_components
madebr Sep 2, 2020
1a91bb9
boost: add more components + correct headers target
madebr Sep 2, 2020
65b2c32
boost: remove boost/1.66-boost/1.68
madebr Sep 2, 2020
ff8bf04
boost: trust on conan's compiler.libcxx
madebr Sep 2, 2020
81b2c56
boost: use headers and _libboost component
madebr Sep 2, 2020
543f3a3
boost: find Python in test_package
madebr Sep 2, 2020
a1ef4e4
boost: attach dependencies to components
madebr Sep 2, 2020
6dec486
boost: extract requirements from boost build tree
madebr Sep 2, 2020
2603c36
boost: remove debug prints
madebr Sep 3, 2020
eeeebe8
boost: add Boost::boost alias of Boost::headers
madebr Oct 8, 2020
b5eaade
Merge remote-tracking branch 'origin/master' into boost_components
madebr Oct 13, 2020
7697c08
Merge remote-tracking branch 'origin/master' into boost_components
madebr Nov 9, 2020
d908efe
boost: fix MSVC build (with default options)
madebr Nov 9, 2020
7abb249
boost: components now support "-o layout=versioned"
madebr Nov 10, 2020
8502b09
boost: fix `-o layout=versioned` on LInux
madebr Nov 11, 2020
f6a509b
boost: add missing configure option + use logging
madebr Nov 12, 2020
8e70962
boost: update boost ymls
madebr Nov 12, 2020
73b0f9b
boost: options from local tuple + requirements from yml + bump icu
madebr Nov 12, 2020
e619e73
boost: add config option check
madebr Nov 12, 2020
ede2270
boost: wrap iostreams compression requirements
madebr Nov 12, 2020
5b6add5
boost: raise a ConanException if no requirements-x.y.z.yml file is found
madebr Nov 13, 2020
5d6178f
boost: set python version in options + don't add python to requiremen…
madebr Nov 13, 2020
7f5ab30
boost: rename _with_xxx to _with_dependency
madebr Nov 15, 2020
c5ba736
boost: only test conan's cmake_find_package
madebr Nov 15, 2020
0c7a4f6
boost: use options.python_executable as path for python executable
madebr Nov 16, 2020
567552e
boost: Add context option + sort values in dependencies yml (for repr…
madebr Dec 2, 2020
516c4a5
boost: use dict.update instead of class for loop
madebr Dec 2, 2020
1a9b7c8
boost: only add defines to components when option enabled
madebr Dec 2, 2020
6f8795f
booost: re-order components in package_info (headers first)
madebr Dec 2, 2020
176a472
boost: remove duplicate settings of component names + move setting of…
madebr Dec 2, 2020
4c73b2a
boost: only make components related to libraries available when build…
madebr Dec 2, 2020
43a8a6b
boost: fix boost:without_python=False
madebr Dec 11, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
144 changes: 100 additions & 44 deletions recipes/boost/all/conanfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
import os
import sys
import shutil
import yaml

try:
from cStringIO import StringIO
Expand Down Expand Up @@ -87,13 +88,51 @@ class BoostConan(ConanFile):
}

for libname in lib_list:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so, we still use hard-coded lib-list here?
probably, we should remove it as well

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indeed, I think it could be used from the yaml

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think conan has a method to do that using external files.
The self.export_folder variable is not available in methods except export.
This is the reason why I copy the ymls to the source folder and copy it to the package folder in package.

The yml must be present in the export folder, because conan should be able to calculate the options/packageid without the sources present.

See conan-io/conan#7283 for my conversation with @memsharded . He also suggested modifying conandata.yml, but that won't work on CCI.
I could always let the script modify the conanfile.py. But I don't like that idea.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay, so we need to modify hook then. @uilianries let's open an issue in hook, but this PR should be good for now until we have hook fixed. this is still an improvement IMO.

if libname != "python":
if libname not in ("graph_parallel", "mpi", "python"):
default_options.update({"without_%s" % libname: False})
default_options.update({"without_python": True})
else:
default_options.update({"without_%s" % libname: True})
short_paths = True
no_copy_source = True
exports_sources = ['patches/*']

@property
def _dependency_filename(self):
return "dependencies-{}.yml".format(self.version)

def export_sources(self):
self.copy(self._dependency_filename, src="dependencies")

_dependencies_cache = None

@property
def _dependencies(self):
if self._dependencies_cache is None:
dep_path = None
if self.source_folder:
dep_src_path = os.path.join(self.source_folder, self._dependency_filename)
if os.path.isfile(dep_src_path):
dep_path = dep_src_path
if not dep_path:
dep_pkg_path = os.path.join(self.package_folder, "lib", self._dependency_filename)
if os.path.isfile(dep_pkg_path):
dep_path = dep_pkg_path
if not dep_path:
raise ConanException("cannot find {}".format(self._dependency_filename))
self._dependencies_cache = yaml.safe_load(open(dep_path))
return self._dependencies_cache

def _all_dependencies_of(self, module):
all_deps = set()
todo = self._dependencies["dependencies"][module]
while todo:
l = todo.pop()
if l in all_deps:
continue
all_deps.add(l)
todo.extend(self._dependencies["dependencies"][l])
return all_deps

@property
madebr marked this conversation as resolved.
Show resolved Hide resolved
def _source_subfolder(self):
return "source_subfolder"
Expand Down Expand Up @@ -140,6 +179,13 @@ def configure(self):
if not self.options.get_safe('without_%s' % lib):
raise ConanInvalidConfiguration("Boost '%s' library requires multi threading" % lib)

def _check_options(self):
for mod_name, mod_deps in self._dependencies["dependencies"].items():
if not self.options.get_safe("without_{}".format(mod_name), False):
for mod_dep in mod_deps:
if self.options.get_safe("without_{}".format(mod_dep), False):
raise ConanInvalidConfiguration("{} requires {}: {} is disabled".format(mod_name, mod_deps, mod_dep))

def build_requirements(self):
self.build_requires("b2/4.2.0")

Expand Down Expand Up @@ -415,6 +461,8 @@ def _run_bcp(self):
self.run(command)

def build(self):
self._check_options()

if self.options.header_only:
self.output.warn("Header only package, skipping build")
return
Expand Down Expand Up @@ -819,10 +867,12 @@ def _toolset(self):
####################################################################

def package(self):
self.copy(self._dependency_filename, src=self.source_folder, dst="lib")
# This stage/lib is in source_folder... Face palm, looks like it builds in build but then
# copy to source with the good lib name
self.copy("LICENSE_1_0.txt", dst="licenses", src=os.path.join(self.source_folder,
self._source_subfolder))

tools.rmdir(os.path.join(self.package_folder, "lib", "cmake"))
if self.options.header_only:
self.copy(pattern="*", dst="include/boost", src="%s/boost" % self._boost_dir)
Expand Down Expand Up @@ -860,80 +910,86 @@ def package_info(self):
major = version_tokens[0]
minor = version_tokens[1]
boost_version_tag = "boost-%s_%s" % (major, minor)
self.cpp_info.includedirs = [os.path.join(self.package_folder, "include", boost_version_tag)]

# List of lists, so if more than one matches the lib like serialization and wserialization
# both will be added to the list
ordered_libs = [[] for _ in range(len(lib_list))]

# The order is important, reorder following the lib_list order
missing_order_info = []
for real_lib_name in gen_libs:
for pos, alib in enumerate(lib_list):
if os.path.splitext(real_lib_name)[0].split("-")[0].endswith(alib):
ordered_libs[pos].append(real_lib_name)
break
else:
# self.output.info("Missing in order: %s" % real_lib_name)
if "_exec_monitor" not in real_lib_name: # https://github.com/bincrafters/community/issues/94
missing_order_info.append(real_lib_name) # Assume they do not depend on other
self.cpp_info.components["headers"].includedirs = [os.path.join(self.package_folder, "include", boost_version_tag)]

self.cpp_info.components["headers"].libs = []

self.cpp_info.components["coroutine"].requires = ["headers"]

detected_libraries = set(tools.collect_libs(self))
used_libraries = set()
for mod_name, mod_deps in self._dependencies["dependencies"].items():
if not self.options.get_safe("without_{}".format(mod_name), False):
dep_libs_all_sorted = self._dependencies["libs"].get(mod_name, [])
dep_libs_avail = set(dep_libs_all_sorted).intersection(detected_libraries)
used_libraries = used_libraries.union(dep_libs_avail)
self.cpp_info.components[mod_name].libs = [l for l in dep_libs_all_sorted if l in dep_libs_avail]

# requires = set(self._dependencies["dependencies"][mod_name]).difference({"mpi", "python"})
requires = set(self._dependencies["dependencies"][mod_name]).intersection(set(self._dependencies["dependencies"].keys()))

self.cpp_info.components[mod_name].requires = list(requires) + ["headers"]
self.cpp_info.components[mod_name].names["cmake_find_package"] = mod_name
self.cpp_info.components[mod_name].names["cmake_find_package_multi"] = mod_name
self.output.info("deps {}: {}".format(mod_name, self.cpp_info.components[mod_name].requires))

# Flat the list and append the missing order
self.cpp_info.libs = [item for sublist in ordered_libs
for item in sublist if sublist] + missing_order_info
if used_libraries != detected_libraries:
non_used = detected_libraries.difference(used_libraries)
raise ConanInvalidConfiguration("These libraries were not used in conan components: {}".format(non_used))

if self.options.without_test: # remove boost_unit_test_framework
self.cpp_info.libs = [lib for lib in self.cpp_info.libs if "unit_test" not in lib]
if self.options.bzip2:
self.cpp_info.components["headers"].requires.append("bzip2::bzip2") # iostreams
if self.options.zlib:
self.cpp_info.components["headers"].requires.append("zlib::zlib") # boost, iostreams

self.output.info("LIBRARIES: %s" % self.cpp_info.libs)
self.output.info("Package folder: %s" % self.package_folder)

# self.cpp_info.components["dynamic_linking"].defines.append("BOOST_ALL_DYN_LINK")
if not self.options.header_only and self.options.shared:
self.cpp_info.defines.append("BOOST_ALL_DYN_LINK")
self.cpp_info.components["headers"].defines.append("BOOST_ALL_DYN_LINK")

if self.options.system_no_deprecated:
self.cpp_info.defines.append("BOOST_SYSTEM_NO_DEPRECATED")
self.cpp_info.components["system"].defines.append("BOOST_SYSTEM_NO_DEPRECATED")

if self.options.asio_no_deprecated:
self.cpp_info.defines.append("BOOST_ASIO_NO_DEPRECATED")
self.cpp_info.components["headers"].defines.append("BOOST_ASIO_NO_DEPRECATED")

if self.options.filesystem_no_deprecated:
self.cpp_info.defines.append("BOOST_FILESYSTEM_NO_DEPRECATED")
self.cpp_info.components["filesystem"].append("BOOST_FILESYSTEM_NO_DEPRECATED")

if self.options.segmented_stacks:
self.cpp_info.defines.extend(["BOOST_USE_SEGMENTED_STACKS", "BOOST_USE_UCONTEXT"])
self.cpp_info.comonents["headers"].extend(["BOOST_USE_SEGMENTED_STACKS", "BOOST_USE_UCONTEXT"])
madebr marked this conversation as resolved.
Show resolved Hide resolved

if self.settings.os != "Android":
if self._gnu_cxx11_abi:
self.cpp_info.defines.append("_GLIBCXX_USE_CXX11_ABI=%s" % self._gnu_cxx11_abi)
self.cpp_info.components["headers"].defines.append("_GLIBCXX_USE_CXX11_ABI=%s" % self._gnu_cxx11_abi)

if not self.options.header_only:
if self.options.error_code_header_only:
self.cpp_info.defines.append("BOOST_ERROR_CODE_HEADER_ONLY")
self.cpp_info.components["headers"].defines.append("BOOST_ERROR_CODE_HEADER_ONLY")

if not self.options.without_python:
if not self.options.shared:
self.cpp_info.defines.append("BOOST_PYTHON_STATIC_LIB")
self.cpp_info.components["python"].defines.append("BOOST_PYTHON_STATIC_LIB")

if self._is_msvc or self._is_clang_cl:
if not self.options.magic_autolink:
# DISABLES AUTO LINKING! NO SMART AND MAGIC DECISIONS THANKS!
self.cpp_info.defines.append("BOOST_ALL_NO_LIB")
self.cpp_info.components["headers"].defines.append("BOOST_ALL_NO_LIB")
self.output.info("Disabled magic autolinking (smart and magic decisions)")
madebr marked this conversation as resolved.
Show resolved Hide resolved
else:
if self.options.layout == "system":
self.cpp_info.defines.append("BOOST_AUTO_LINK_SYSTEM")
self.cpp_info.components["headers"].defines.append("BOOST_AUTO_LINK_SYSTEM")
elif self.options.layout == "tagged":
self.cpp_info.defines.append("BOOST_AUTO_LINK_TAGGED")
self.cpp_info.components["headers"].defines.append("BOOST_AUTO_LINK_TAGGED")
self.output.info("Enabled magic autolinking (smart and magic decisions)")

# https://github.com/conan-community/conan-boost/issues/127#issuecomment-404750974
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This link is broken

self.cpp_info.system_libs.append("bcrypt")
self.cpp_info.components["headers"].system_libs.append("bcrypt")
elif self.settings.os == "Linux":
# https://github.com/conan-community/community/issues/135
self.cpp_info.system_libs.append("rt")
self.cpp_info.components["headers"].system_libs.append("rt")
if self.options.multithreading:
self.cpp_info.system_libs.append("pthread")
self.cpp_info.components["headers"].system_libs.append("pthread")
elif self.settings.os == "Emscripten":
if self.options.multithreading:
arch = self.settings.get_safe('arch')
Expand All @@ -943,11 +999,11 @@ def package_info(self):
# So instead we are using the raw compiler flags (that are being activated
# from the aformentioned flag)
if arch.startswith("x86") or arch.startswith("wasm"):
self.cpp_info.cxxflags.append("-pthread")
self.cpp_info.sharedlinkflags.extend(["-pthread","--shared-memory"])
self.cpp_info.exelinkflags.extend(["-pthread","--shared-memory"])
self.cpp_info.components["headers"].cxxflags.append("-pthread")
self.cpp_info.components["headers"].sharedlinkflags.extend(["-pthread","--shared-memory"])
self.cpp_info.components["headers"].exelinkflags.extend(["-pthread","--shared-memory"])

self.env_info.BOOST_ROOT = self.package_folder
self.cpp_info.bindirs.append("lib")
self.cpp_info.components["headers"].bindirs.append("lib")
self.cpp_info.names["cmake_find_package"] = "Boost"
self.cpp_info.names["cmake_find_package_multi"] = "Boost"
Loading