art:build-info create fails for header-library

[CJCombrink]

Trying to run conan art:build-info create on my package fails with the following stacktrace

ERROR: Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/conan/cli/cli.py", line 307, in main
    cli.run(args)
  File "/usr/local/lib/python3.9/site-packages/conan/cli/cli.py", line 192, in run
    command.run(self._conan_api, args[0][1:])
  File "/usr/local/lib/python3.9/site-packages/conan/cli/command.py", line 196, in run
    sub.run(conan_api, parser, *args)
  File "/usr/local/lib/python3.9/site-packages/conan/cli/command.py", line 214, in run
    info = self._method(conan_api, parent_parser, self._parser, *args)
  File "/home/Buildit/.conan2/extensions/commands/art/cmd_build_info.py", line 398, in build_info_create
    cli_out_write(bi.create())
  File "/home/Buildit/.conan2/extensions/commands/art/cmd_build_info.py", line 309, in create
    bi.update({"modules": self.get_modules()})
  File "/home/Buildit/.conan2/extensions/commands/art/cmd_build_info.py", line 279, in get_modules
    deps_artifacts = self.get_artifacts(nodes.get(require_id), "package",
  File "/home/Buildit/.conan2/extensions/commands/art/cmd_build_info.py", line 212, in get_artifacts
    artifacts, missing = _get_local_artifacts()
  File "/home/Buildit/.conan2/extensions/commands/art/cmd_build_info.py", line 145, in _get_local_artifacts
    artifacts_folder = Path(node.get("package_folder")) if artifact_type == "package" else Path(node.get("recipe_folder"))
  File "/usr/lib64/python3.9/pathlib.py", line 1082, in __new__
    self = cls._from_parts(args, init=False)
  File "/usr/lib64/python3.9/pathlib.py", line 707, in _from_parts
    drv, root, parts = self._parse_args(args)
  File "/usr/lib64/python3.9/pathlib.py", line 691, in _parse_args
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

ERROR: expected str, bytes or os.PathLike object, not NoneType

The package that triggers this is the sqlpp11 package.

sqlpp11 dictionary dump

{
    'ref': 'sqlpp11/0.63#46f6808e3c740b6b4f5e617b18534478',
    'id': '105',
    'recipe': 'Cache',
    'package_id': 'da39a3ee5e6b4b0d3255bfef95601890afd80709',
    'prev': '027d58ae8e43bc52075d46f449e22588',
    'rrev': '46f6808e3c740b6b4f5e617b18534478',
    'rrev_timestamp': 1731681636.746,
    'prev_timestamp': 1731681636.843,
    'remote': None,
    'binary_remote': 'vt_conan',
    'build_id': None,
    'binary': 'Skip',
    'invalid_build': False,
    'info_invalid': None,
    'name': 'sqlpp11',
    'user': None,
    'channel': None,
    'url': 'https://github.com/conan-io/conan-center-index',
    'license': 'BSD-2-Clause',
    'author': None,
    'description': 'A type safe SQL template library for C++',
    'homepage': 'https://github.com/rbock/sqlpp11',
    'build_policy': None,
    'upload_policy': None,
    'revision_mode': 'hash',
    'provides': None,
    'deprecated': None,
    'win_bash': None,
    'win_bash_run': None,
    'default_options': None,
    'options_description': None,
    'version': '0.63',
    'topics': ['sql', 'dsl', 'embedded', 'data-base'],
    'package_type': 'header-library',
    'languages': [],
    'settings': {
        'os': 'Linux',
        'os.distro': 'centos',
        'arch': 'x86_64',
        'compiler': 'gcc',
        'compiler.cppstd': '20',
        'compiler.libcxx': 'libstdc++11',
        'compiler.version': '12',
        'build_type': 'Release'
    },
    'options': {},
    'options_definitions': {},
    'generators': [],
    'python_requires': None,
    'system_requires': {},
    'recipe_folder': '/home/Buildit/.conan2/p/sqlpp348e5562d2f74/e',
    'source_folder': None,
    'build_folder': None,
    'generators_folder': None,
    'package_folder': None,
    'immutable_package_folder': None,
    'cpp_info': {
        'root': {
            'includedirs': ['include'],
            'srcdirs': None,
            'libdirs': ['lib'],
            'resdirs': None,
            'bindirs': ['bin'],
            'builddirs': None,
            'frameworkdirs': None,
            'system_libs': None,
            'frameworks': None,
            'libs': None,
            'defines': None,
            'cflags': None,
            'cxxflags': None,
            'sharedlinkflags': None,
            'exelinkflags': None,
            'objects': None,
            'sysroot': None,
            'requires': None,
            'properties': None,
            'exe': None,
            'type': None,
            'location': None,
            'link_location': None,
            'languages': None
        }
    },
    'conf_info': {},
    'label': 'sqlpp11/0.63',
    'info': {},
    'vendor': False,
    'dependencies': {
        '106': {
            'ref': 'date/3.0.1',
            'run': False,
            'libs': False,
            'skip': False,
            'test': False,
            'force': False,
            'direct': True,
            'build': False,
            'transitive_headers': True,
            'transitive_libs': True,
            'headers': True,
            'package_id_mode': 'unrelated_mode',
            'visible': True
        }
    },
    'context': 'host',
    'test': False
}

I did some printing and this is what I see:

package_folder: None
artifact_type: package
recipe_folder: /home/Buildit/.conan2/p/sqlpp348e5562d2f74/e

Thus the check in _get_local_artifacts resolves to None since artifact_type == "package" but it does not have a package_folder

Any help will be appreciated

Edit: Forgot to mention
Conan version 2.10.1 (also tested on 2.10.3)
sqlpp11 is an indirect dependency of my library
It appears that this issue is an duplicate of: #165

[czoido]

Hi @CJCombrink,

Thanks for reporting.

A couple of questions:

• Are you using the latest version of the extensions? Please make sure you are doing: conan config install https://github.com/conan-io/conan-extensions.git
• Can you provide more details on the sequence of steps that make you get that error?

[CJCombrink]

Are you using the latest version of the extensions?

Yes, my CI pipeline installs from scratch each time (which is problem for another day). Also to reproduce locally I had to install it now.

Can you provide more details on the sequence of steps that make you get that error?

These are my steps to reproduce:

> conan --version #Conan version 2.10.3
> conan remove '*' --confirm
> conan install .. --user <user> --channel testing -s '&:build_type=Release' --build missing --lockfile-out conan.lock --format=json > install.json
> conan create .. --user <user> --channel testing --lockfile conan.lock --build missing --format json > create.json
> conan art:build-info create create.json "<info>" "5" '<repo>' --server <server> --with-dependencies

Just a comment on the steps:
The install before create is done because of our CI pipeline
- Trying to replicate exactly what CI does
- This is still pretty much our conan 1 pipelines that we will revisit after the conan 2 migration is done
- The install creates the lockfile that create uses

As mentioned, my package depends on a package that depends on sqlpp11

LibA

    ... 
    def requirements(self):
        ....
        self.requires("sqlpp11/0.63")
        self.requires("libpq/14.7")

  def package_info(self):
        self.cpp_info.set_property("cmake_file_name", "<NAME>")

        self.cpp_info.components["lib"].set_property("cmake_target_name", "<NAME>::database")
        self.cpp_info.components["lib"].libs = ["database"]
        self.cpp_info.components["lib"].requires = [
                                                    'sqlpp11::sqlpp11',
                                                    'libpq::libpq',
                                                     ,,, ]

LibB (this one)

    def requirements(self):
        ... 
        # Internal
        self.requires("database/[>=8.2]@<channel>/stable")

I hope this helps

[CJCombrink]

I have now marked sqlpp11 as hidden in LibA but now I get the same failure for libpq which is not a header only library

artifact_type : package
recipe_folder : /home/Buildit/.conan2/p/libpqc0db551649ead/e
package_folder: None

libpq node dump

{
    "ref": "libpq/14.7#171c3ba1fadc7ae7ac9c9c9befd9958c",
    "id": "107",
    "recipe": "Downloaded",
    "package_id": "91d3c3993cdc92e040e5df50312132e34bddb616",
    "prev": "4350c8dea8a4e3fcb22d3ca62fe92133",
    "rrev": "171c3ba1fadc7ae7ac9c9c9befd9958c",
    "rrev_timestamp": 1731681629.902,
    "prev_timestamp": 1733992540.285,
    "remote": "vt_conan",
    "binary_remote": "vt_conan",
    "build_id": None,
    "binary": "Skip",
    "invalid_build": False,
    "info_invalid": None,
    "name": "libpq",
    "user": None,
    "channel": None,
    "url": "https://github.com/conan-io/conan-center-index",
    "license": "PostgreSQL",
    "author": None,
    "description": "The library used by all the standard PostgreSQL tools.",
    "homepage": "https://www.postgresql.org/docs/current/static/libpq.html",
    "build_policy": None,
    "upload_policy": None,
    "revision_mode": "hash",
    "provides": None,
    "deprecated": None,
    "win_bash": None,
    "win_bash_run": None,
    "default_options": {
        "shared": False,
        "fPIC": True,
        "with_openssl": False,
        "disable_rpath": False
    },
    "options_description": None,
    "version": "14.7",
    "topics": ["libpq", "postgresql", "database", "db"],
    "package_type": "static-library",
    "languages": [],
    "settings": {
        "os": "Linux",
        "os.distro": "centos",
        "arch": "x86_64",
        "compiler": "gcc",
        "compiler.version": "12",
        "build_type": "Release"
    },
    "options": {
        "disable_rpath": "False",
        "fPIC": "True",
        "shared": "False",
        "with_openssl": "False"
    },
    "options_definitions": {
        "shared": ["True", "False"],
        "fPIC": ["True", "False"],
        "with_openssl": ["True", "False"],
        "disable_rpath": ["True", "False"]
    },
    "generators": [],
    "python_requires": None,
    "system_requires": {},
    "recipe_folder": "/home/Buildit/.conan2/p/libpqc0db551649ead/e",
    "source_folder": None,
    "build_folder": None,
    "generators_folder": None,
    "package_folder": None,
    "immutable_package_folder": None,
    "cpp_info": {
        "root": {
            "includedirs": ["include"],
            "srcdirs": None,
            "libdirs": ["lib"],
            "resdirs": None,
            "bindirs": ["bin"],
            "builddirs": None,
            "frameworkdirs": None,
            "system_libs": None,
            "frameworks": None,
            "libs": None,
            "defines": None,
            "cflags": None,
            "cxxflags": None,
            "sharedlinkflags": None,
            "exelinkflags": None,
            "objects": None,
            "sysroot": None,
            "requires": None,
            "properties": None,
            "exe": None,
            "type": None,
            "location": None,
            "link_location": None,
            "languages": None
        }
    },
    "conf_info": {},
    "label": "libpq/14.7",
    "info": {
        "settings": {
            "os": "Linux",
            "os.distro": "centos",
            "arch": "x86_64",
            "compiler": "gcc",
            "compiler.version": "12",
            "build_type": "Release"
        },
        "options": {
            "disable_rpath": "False",
            "fPIC": "True",
            "shared": "False",
            "with_openssl": "False"
        }
    },
    "vendor": False,
    "dependencies": {},
    "context": "host",
    "test": False
}

[CJCombrink]

I have now upgraded to the latest conan but it does not make any difference

I am able to make progress by specifying to build the specific packages as part of the create step

> conan remove '*' --confirm && rm -fr ../build/*
> conan create .. --user <user> --channel testing --build=<LibB> --build='sqlpp11/*' --build='libpq/*' --build missing --format json > create.json
> conan art:build-info create create.json "<info>" "5" '<repo>' --server <server> --with-dependencies

PS: I have dropped the install step from my local testing

It might be that this is a bug in conan instead of this repo?

[CJCombrink]

Also getting the same in another package but this time nlohmann_json

artifact_type : package
recipe_folder : /home/Buildit/.conan2/p/nlohm8cff3addf075b/e
package_folder: None

nlohmann_json node dump

{
    "ref": "nlohmann_json/3.11.3#45828be26eb619a2e04ca517bb7b828d",
    "id": "60",
    "recipe": "Downloaded",
    "package_id": "da39a3ee5e6b4b0d3255bfef95601890afd80709",
    "prev": "552828a2560333bcbeff9d087ce0af0f",
    "rrev": "45828be26eb619a2e04ca517bb7b828d",
    "rrev_timestamp": 1732256181.932,
    "prev_timestamp": 1732256182.08,
    "remote": "vt_conan",
    "binary_remote": "vt_conan",
    "build_id": None,
    "binary": "Skip",
    "invalid_build": False,
    "info_invalid": None,
    "name": "nlohmann_json",
    "user": None,
    "channel": None,
    "url": "https://github.com/conan-io/conan-center-index",
    "license": "MIT",
    "author": None,
    "description": "JSON for Modern C++ parser and generator.",
    "homepage": "https://github.com/nlohmann/json",
    "build_policy": None,
    "upload_policy": None,
    "revision_mode": "hash",
    "provides": None,
    "deprecated": None,
    "win_bash": None,
    "win_bash_run": None,
    "default_options": None,
    "options_description": None,
    "version": "3.11.3",
    "topics": ["json", "header-only"],
    "package_type": "header-library",
    "languages": [],
    "settings": {
        "os": "Linux",
        "os.distro": "centos",
        "arch": "x86_64",
        "compiler": "gcc",
        "compiler.cppstd": "20",
        "compiler.libcxx": "libstdc++11",
        "compiler.version": "12",
        "build_type": "Release"
    },
    "options": {},
    "options_definitions": {},
    "generators": [],
    "python_requires": None,
    "system_requires": {},
    "recipe_folder": "/home/Buildit/.conan2/p/nlohm8cff3addf075b/e",
    "source_folder": None,
    "build_folder": None,
    "generators_folder": None,
    "package_folder": None,
    "immutable_package_folder": None,
    "cpp_info": {
        "root": {
            "includedirs": ["include"],
            "srcdirs": None,
            "libdirs": ["lib"],
            "resdirs": None,
            "bindirs": ["bin"],
            "builddirs": None,
            "frameworkdirs": None,
            "system_libs": None,
            "frameworks": None,
            "libs": None,
            "defines": None,
            "cflags": None,
            "cxxflags": None,
            "sharedlinkflags": None,
            "exelinkflags": None,
            "objects": None,
            "sysroot": None,
            "requires": None,
            "properties": None,
            "exe": None,
            "type": None,
            "location": None,
            "link_location": None,
            "languages": None
        }
    },
    "conf_info": {},
    "label": "nlohmann_json/3.11.3",
    "info": {},
    "vendor": False,
    "dependencies": {},
    "context": "host",
    "test": False
}

[danimtb]

Hi @CJCombrink:

I looked at the information provided and although I haven't reproduced the issue locally, I have made a few tries and the issue might come from the package type of the transitive dependencies. Let me explain:

The packages reported (sqlpp11/0.63#46f6808e3c740b6b4f5e617b18534478, libpq/14.7#171c3ba1fadc7ae7ac9c9c9befd9958c then and nlohmann_json/3.11.3#45828be26eb619a2e04ca517bb7b828d) have something in common in the graph: They are marked as "binary": "Skip". This means that Conan didn't need to download the binary package from the remote for some reason.

In the case of libpq/14.7#171c3ba1fadc7ae7ac9c9c9befd9958c, its package type is "package_type": "static-library", so it makes sense as the resulting binary will be embedded (it is a transitive dependency) in the library that requires it down the dependency graph.

For sqlpp11/0.63#46f6808e3c740b6b4f5e617b18534478 and nlohmann_json/3.11.3#45828be26eb619a2e04ca517bb7b828d they are marked as "package_type": "header-library". I am not quite sure about the Conan behavior in this case but I think a similar thing could be happening: They are embedded into a requirement down in the graph (as they both are transitive dependencies as well).

I could not find a good reference in the documentation explaining this "skip" download binary behavior, only this example at https://docs.conan.io/2/ci_tutorial/products_pipeline/multi_product.html

as mapviewer is an application linked statically, Conan knows that it can “skip” its dependencies binaries.

And the reference at https://docs.conan.io/2/reference/conanfile/methods/requirements.html#package-type-trait-inferring does not help much with that.

So yeah, it is not straightforward to understand behavior and the solution for the build info is not that clear to me, so I will need to discuss a possible fix internally.

Thanks again for the report. I will come back (hopefully) with a fix! 😄

[CJCombrink]

@danimtb

Thanks for reporting back. I will be on holiday util after Jan 6 so will not need or be able to provide updates until then.
Perhaps after that I can provide more info if needed.

[danimtb]

@CJCombrink thank you for the heads-up.

Anyway, while we work on a fix, you can add this configuration entry to the global conan conf so binary downloads are always forced tools.graph:skip_binaries=False

See for reference: https://docs.conan.io/2/reference/config_files/global_conf.html

You can also use it in the command line if you just want to apply it only for that command:

$ conan create ... -c tools.graph:skip_binaries=False

[dnoveczky-ntr]

Hey,

@danimtb
We've been seeing the same issue in our project, I looked into the issue and came up with a fix for it:
#168
Merge please if it looks good to you!

Thanks

[CJCombrink]

A proper fix would be good (I have not checked the PR yet)

Just want to mention the suggested workaround of c tools.graph:skip_binaries=False does work as mention

[dnoveczky-ntr]

A proper fix would be good (I have not checked the PR yet)

Just want to mention the suggested workaround of c tools.graph:skip_binaries=False does work as mention

Could you please test my changes? You can just overwrite ~\.conan2\extensions\commands\art\cmd_build_info.py with the one from my PR. Hopefully it will help Daniel with the review knowing that it works for others too or not.

It's good to know the workaround works, but I'm not a big fan of force downloading unnecessary things as we are trying to reduce unnecessary traffic as much in our project as we can, costs can go up quickly with bigger projects and frequent CI builds...

[danimtb]

Hi @dnoveczky-ntr,

Your error described at #168 (comment) is a similar case as the one described here but the solution proposed is based on a bad diagnosis IMO. The recipe in the log is Meson, and it is not a recipe-only node, but an actual package that packages files (at least the recipe in conan center https://github.com/conan-io/conan-center-index/blob/master/recipes/meson/all/conanfile.py).

However, if you look at the JSON file log you posted there, the Meson node is also marked as binary: skip, and therefore, the package folder is not present in the cache which leads to the error of this issue. I am not sure why the meson binary is skipped in your case, Conan probably downloads the recipe for some reason but the package of Meson is not needed for building. Maybe your meson tool requirement affects the package ID of your dependencies? If you could post a complete JSON with the meson node it would be helpful to debug this case as well.

Anyway, we will try to fix the issue for all the skip binary cases in PR #167 (still not finished).
Thanks a lot for the report and the effort in proposing a fix in your PR!

[dnoveczky-ntr]

Hey @danimtb,

Thanks for taking the time to look at the issue.
I created a minimal repro that will hopefully help with the investigation: https://github.com/dnoveczky-ntr/conan-build-info-bug-repro
And this is the .json file that's the input for the failing build info generation:
conan_build_info.json
I'm not super familiar with the inner workings of conan, but probably what's happening is that meson is a transitive dependency, but only a build dependency, so its binaries are not needed to build the consumer of freetype, so they are skipped. So I think it's not needed in the build info file either. So maybe the proper fix would be to just skip the package if it is marked as binary: skipped What do you think?

Thanks again for taking the time to deal with this issue!

[CJCombrink]

@czoido @danimtb
Does this mean I can now drop the -c tools.graph:skip_binaries=False part from my CI or is it still advised to keep this?
I will try to test first thing tomorrow

[danimtb]

@czoido @danimtb Does this mean I can now drop the -c tools.graph:skip_binaries=False part from my CI or is it still advised to keep this? I will try to test first thing tomorrow

It means that the packages that are skipped will not be added to the build info as they are not available in the cache. If you want to force the include, you would have to use -c:a tools.graph:skip_binaries=False.

We believe this is the best trade-off to solve this issue. Thank you!