Unexpected dependencies when building docker image

[torgeirsh]

I have the following default.nix:

let
    nixpkgs = import (fetchTarball https://github.com/nixos/nixpkgs/archive/21.11.tar.gz) {};
in
    nixpkgs.haskellPackages.callCabal2nix "test" ./. {}

And test.cabal looks like this:

cabal-version: 1.12

name: test
version: 0.1.0.0

library lib
    exposed-modules: Lib
    build-depends: base

executable test
    main-is: Main.hs
    build-depends: base, lib

Then I use docker.nix to build a docker image:

let
    nixpkgs = import (fetchTarball https://github.com/nixos/nixpkgs/archive/21.11.tar.gz) {};
in nixpkgs.dockerTools.buildLayeredImage {
    name = "test";
    contents = import ./default.nix;
}

The resulting docker image is almost 400 MB, and contains ghc, gcc, documentation, and so on. However, if I remove the library from the cabal file, the resulting docker image is only 14 MB, and doesn't contain anything superfluous. I guess that when the cabal file contains a library, it becomes part of the build result, and since a library isn't useful on its own, it depends on ghc? However, I don't want to make the library available, it's only for building the executable. How can I get a lean docker image without a build environment, while keeping the library structure of the cabal project?

[sternenseemann]

You can use haskell.lib.compose.justStaticExecutables to achieve this which will discard everything except the binary which links against all Haskell dependencies statically.

The library needs to reference e. g. GHC as you need all libraries the library when linking against the library and base is bundled with GHC.

[torgeirsh]

Thank you! If you don't mind, I would like to get a better understanding of why static linking is necessary. Using the following cabal file, the result doesn't depend on ghc, even though I don't use justStaticExecutables:

cabal-version: 1.12

name: test
version: 0.1.0.0

executable test
    main-is: Main.hs
    build-depends: base

You mention that base is bundled with ghc, but here I'm able to link to base without incurring a dependency on ghc. I'm confused about why this situation changes when a library is added to the cabal file, and why static linking is the issue.

[sternenseemann]

All executables are statically linked against all their Haskell dependencies (base, … and even the library defined in your package). Static linking means all necessary code coming from the libraries is copied into the executable, instead of loading it dynamically at runtime from the original files. Thus when something is statically linked, we only need its dependencies at build time, not at run time.

For libraries you can't copy everything necessary into the library, since then you'll run into issues with the same function being defined in multiple libraries (e. g. head from base would be copied virtually everywhere, when building something you'd have a thousand conflicting heads). Therefore every library only contains the code it actually defines itself and we propagate all its dependencies, since when we later build an executable, we'll also need some code from dependencies of our dependencies into the executable.

In nixpkgs, executables are always statically linked, so naturally a cabal package without a library won't propagate GHC. All justStaticExecutables does is delete the lib directory from the output and disabling propagation of dependencies which accounts for the unwanted references. It doesn't change how the package is built per se.

[torgeirsh]

That's a great explanation, and I could verify that deleting the library is the only necessary change to default.nix:

let
    nixpkgs = import (fetchTarball https://github.com/nixos/nixpkgs/archive/21.11.tar.gz) { };
in
    nixpkgs.haskell.lib.overrideCabal (nixpkgs.haskellPackages.callCabal2nix "test" ./. { }) (drv: {
        postFixup = drv.postFixup or "" + "rm -rf $out/lib";
    })

It makes me wonder if there's a way to not include the library to begin with. Libraries in cabal files are internal if they're named, but using the name "lib" in my example didn't prevent it from being included. Is that perhaps a bug in how Nix builds cabal projects?

[sternenseemann]

It makes me wonder if there's a way to not include the library to begin with. Libraries in cabal files are internal if they're named, but using the name "lib" in my example didn't prevent it from being included. Is that perhaps a bug in how Nix builds cabal projects?

It appears so, I've filed NixOS/cabal2nix#539 for now. It does, however, not really matter in most cases that are of concern to us (pure executable packages need manual intervention anyways most of the time and the rest we ship is mostly libraries), so I don't think it's a particularly urgent issue.

I'll close this issue, since everything described is working as intended/expected, but do feel free to ask further questions in this thread if you have any.

[m4dc4p]

I really think this issue should remain open, at least to help others know it's a problem.

I'm dealing with this problem right now, and wouldn't have seen any of this if I didn't decide to look through closed issues as well as open.

[m4dc4p]

@torgeirsh I also found this issue re-appears if the library uses & imports any data files (by importing the cabal-generated Paths_<package> module).

[torgeirsh]

That's good to know. Can you work around it in the same way, by deleting files in postFixup?

[m4dc4p]

I don't think so. Nix finds a reference to the library in the executable produced. xxx-core is the executable in the below; xxx-shared is a shared library (containing the data files):

$ nix why-depends ...-xxxx-core-0.0.0  ...-ghc-8.10.7
/nix/store/hvx06a9cnijqf4pqq9s8lvbz8ncfbr15-xxxx-core-0.0.0
└───bin/xxxx-core: …xxxx_shared_datadir./nix/store/ai5ncjrwa91x6q9m44p2hykbs5rib86w-xxxx-shared-0.0.0/share/x86_…
    → /nix/store/ai5ncjrwa91x6q9m44p2hykbs5rib86w-xxxx-shared-0.0.0
    └───lib/ghc-8.10.7/package.conf.d/xxxx-shared-0.0.0-4PTUc3LgwErFQdModM8LE.conf: …p5-gmp-6.2.1/lib.    /nix/store/86hjrkd6c14lq05ysspmp5qn02pa7jvr-ghc-8.10.7/lib..dynamic-library…
        → /nix/store/86hjrkd6c14lq05ysspmp5qn02pa7jvr-ghc-8.10.7

Right now, I think the only way to avoid this is not use the data-files feature.