uv: Build as a standalone without Python #357113
Conversation
adisbladis
force-pushed
the
uv-standalone
branch
from
6bfb982 to
730d153
Compare
adisbladis
force-pushed
the
uv-standalone
branch
from
730d153 to
09d02b9
Compare
|
LGTM. Since the latest updates of uv unfortunately went to staging, we should target staging for this PR so that the staging merge to master doesn't result in merge conflicts. |
adisbladis
force-pushed
the
uv-standalone
branch
from
09d02b9 to
5311f74
Compare
Using `buildPythonPackage` make `ruff` propagate the Python used for the build, which might not be the same Python as you want in your development shell. NixOS#350654 changed the ruff top-level attribute to be built with Python modules. This doesn't actually make much sense, we have versioned Python sets, and including just the one Python in the top-level package isn't helpful, and causes issues downstream related to PATH & PYTHONPATH. See my related [uv PR](NixOS#357113 (comment)).
Using `buildPythonPackage` make `ruff` propagate the Python used for the build, which might not be the same Python as you want in your development shell. NixOS#350654 changed the ruff top-level attribute to be built with Python modules. This doesn't actually make much sense, we have versioned Python sets, and including just the one Python in the top-level package isn't helpful, and causes issues downstream related to PATH & PYTHONPATH. See my related [uv PR](NixOS#357113 (comment)).
adisbladis
force-pushed
the
uv-standalone
branch
2 times, most recently
from
acfc901 to
c011a4d
Compare
|
Thank you for submitting this change. I was quite skeptical about #350654 but didn't have a better idea at the time. |
|
I was able to build this on apple silicon by checking out the pull request and running I would like to check that this pull request works on apple silicon - can somebody give me a hint how I can do so without having a compile farm? |
This is expected. Maybe you haven't noticed, but this PR now targets the |
|
@GaetanLepage Thanks! I am still quite new to Nixpkgs and this is something I did not know yet. Could you recommend a different way in which I could contribute the fact that this works on Darwin? |
There was a problem hiding this comment.
See comment in the ruff package
#358029 (review)
But same idea here, you're creating duplication and going astray of what upstream intends. This will cause issues further down the line.
|
If you want to strip off the propagation, you can do so in your nix-shell: |
|
I would be glad to have the opinion of more experienced maintainers on this matter. |
This isn't just problematic for development shells, but also for builds. I'm using uv in low level packaging scripts (replacing all the nixpkgs python build infra), and Uv isn't a Python package. It's a Rust package to build Python.
Propagation is only part of the problem, pulling in a Python needlessly is a problem in itself. |
We shouldn't include python when it isn't a build requirement, especially when these tools not only can, but also are commonly be used without python at all. You can just follow other package's lead and have a top level nixpkgs/pkgs/top-level/python-packages.nix Lines 18295 to 18297 in d92e4e2 |
Using `buildPythonPackage` make `ruff` propagate the Python used for the build, which might not be the same Python as you want in your development shell. NixOS#350654 changed the ruff top-level attribute to be built with Python modules. This doesn't actually make much sense, we have versioned Python sets, and including just the one Python in the top-level package isn't helpful, and causes issues downstream related to PATH & PYTHONPATH. See my related [uv PR](NixOS#357113 (comment)).
Using `buildPythonPackage` make `ruff` propagate the Python used for the build, which might not be the same Python as you want in your development shell. NixOS#350654 changed the ruff top-level attribute to be built with Python modules. This doesn't actually make much sense, we have versioned Python sets, and including just the one Python in the top-level package isn't helpful, and causes issues downstream related to PATH & PYTHONPATH. See my related [uv PR](NixOS#357113 (comment)).
|
An alternative approach would be to make multiple outputs: --- a/pkgs/by-name/uv/uv/package.nix
+++ b/pkgs/by-name/uv/uv/package.nix
@@ -22,6 +22,11 @@ python3Packages.buildPythonApplication rec {
hash = "sha256-xy/fgy3+YvSdfq5ngPVbAmRpYyJH27Cft5QxBwFQumU=";
};
+ outputs = [
+ "bin"
+ "out"
+ ];
+
cargoDeps = rustPlatform.importCargoLock {
lockFile = ./Cargo.lock;
outputHashes = {
@@ -48,11 +53,15 @@ python3Packages.buildPythonApplication rec {
];
postInstall = ''
+ mkdir -p $bin/bin
+ mv $out/bin/uv $out/bin/uvx $bin/bin/
+ rmdir $out/bin
+
export HOME=$TMPDIR
installShellCompletion --cmd uv \
- --bash <($out/bin/uv --generate-shell-completion bash) \
- --fish <($out/bin/uv --generate-shell-completion fish) \
- --zsh <($out/bin/uv --generate-shell-completion zsh)
+ --bash <($bin/bin/uv --generate-shell-completion bash) \
+ --fish <($bin/bin/uv --generate-shell-completion fish) \
+ --zsh <($bin/bin/uv --generate-shell-completion zsh)
'';
pythonImportsCheck = [ "uv" ];You don't get the with import <nixpkgs> {};
mkShell {
buildInputs = [ ruff.bin uv.bin ];
}Their dependency tree: the nixpkgs/pkgs/by-name/ru/ruff/package.nix Line 36 in d440628 |
|
In the case of uv(though not ruff), |
|
And the one just above relying on wheel and maturin? Their |
|
My point is that:
Of course those can be worked around, but they can be quite annoying and difficult to understand. So, given that there is a reasonable way to package that avoids those problems, it makes sense to me to just avoid those problems. |
|
Okay. Have you considered the suggestion I've made just above then? What about splitting the outputs? Because of the ordering choice I'm making, by default you would only get the binaries. We retain the single binary output, they don't bring in python, and we keep the original maturin build? |
Using `buildPythonPackage` triggers dependency propagation, meaning that using `uv` with `nix-shell` results in a a shell with the `uv` Python input in it. This is problematic for development usage where you only want the one specified version. Often this design bug in the Python package builders is worked around by deleting `$out/nix-support/propagated-build-inputs`, but since `uv` is written in Rust and can be built without a Python interpreter so it's better to just build without a Python interpreter.
adisbladis
force-pushed
the
uv-standalone
branch
from
c011a4d to
7b67d15
Compare
|
It think splitting outputs also makes sense |
No that doesn't make sense. Even bringing an unrelated Python into the build closure is hugely problematic. |
|
As far as I know uv is a build tool written in Rust with no python dependencies so I would agree with @adisbladis trying to do in this PR. Thought I add my two cents FWIW in case helpful in reaching a positive outcome. |
So just to make sure I understand, what you're railing against is the following dependency graph: And instead what you expect is: If
What is your opinion about using Should we remove As far as I can tell, this looks fairly acceptable. |
|
I don't actually care, I removed my block. I'd still like to see more experienced maintainers opinion on this before this merges |
|
Thank you! |
Using `buildPythonPackage` make `ruff` propagate the Python used for the build, which might not be the same Python as you want in your development shell. NixOS#350654 changed the ruff top-level attribute to be built with Python modules. This doesn't actually make much sense, we have versioned Python sets, and including just the one Python in the top-level package isn't helpful, and causes issues downstream related to PATH & PYTHONPATH. See my related [uv PR](NixOS#357113 (comment)).
Using `buildPythonPackage` make `ruff` propagate the Python used for the build, which might not be the same Python as you want in your development shell. NixOS#350654 changed the ruff top-level attribute to be built with Python modules. This doesn't actually make much sense, we have versioned Python sets, and including just the one Python in the top-level package isn't helpful, and causes issues downstream related to PATH & PYTHONPATH. See my related [uv PR](NixOS#357113 (comment)).
Using `buildPythonPackage` make `ruff` propagate the Python used for the build, which might not be the same Python as you want in your development shell. NixOS#350654 changed the ruff top-level attribute to be built with Python modules. This doesn't actually make much sense, we have versioned Python sets, and including just the one Python in the top-level package isn't helpful, and causes issues downstream related to PATH & PYTHONPATH. See my related [uv PR](NixOS#357113 (comment)).
Using `buildPythonPackage` make `ruff` propagate the Python used for the build, which might not be the same Python as you want in your development shell. NixOS#350654 changed the ruff top-level attribute to be built with Python modules. This doesn't actually make much sense, we have versioned Python sets, and including just the one Python in the top-level package isn't helpful, and causes issues downstream related to PATH & PYTHONPATH. See my related [uv PR](NixOS#357113 (comment)).
Using `buildPythonPackage` make `ruff` propagate the Python used for the build, which might not be the same Python as you want in your development shell. NixOS#350654 changed the ruff top-level attribute to be built with Python modules. This doesn't actually make much sense, we have versioned Python sets, and including just the one Python in the top-level package isn't helpful, and causes issues downstream related to PATH & PYTHONPATH. See my related [uv PR](NixOS#357113 (comment)).
Using `buildPythonPackage` make `ruff` propagate the Python used for the build, which might not be the same Python as you want in your development shell. NixOS#350654 changed the ruff top-level attribute to be built with Python modules. This doesn't actually make much sense, we have versioned Python sets, and including just the one Python in the top-level package isn't helpful, and causes issues downstream related to PATH & PYTHONPATH. See my related [uv PR](NixOS#357113 (comment)).
Using `buildPythonPackage` make `ruff` propagate the Python used for the build, which might not be the same Python as you want in your development shell. NixOS#350654 changed the ruff top-level attribute to be built with Python modules. This doesn't actually make much sense, we have versioned Python sets, and including just the one Python in the top-level package isn't helpful, and causes issues downstream related to PATH & PYTHONPATH. See my related [uv PR](NixOS#357113 (comment)).
Using `buildPythonPackage` make `ruff` propagate the Python used for the build, which might not be the same Python as you want in your development shell. NixOS#350654 changed the ruff top-level attribute to be built with Python modules. This doesn't actually make much sense, we have versioned Python sets, and including just the one Python in the top-level package isn't helpful, and causes issues downstream related to PATH & PYTHONPATH. See my related [uv PR](NixOS#357113 (comment)).
Things done
Using
buildPythonPackagetriggers dependency propagation, meaning that usinguvwithnix-shellresults in a a shell with theuvPython input in it.This is problematic for development usage where you only want the one specified version.
Often this design bug in the Python package builders is worked around by deleting
$out/nix-support/propagated-build-inputs, but sinceuvis written in Rust and can be built without a Python interpreter so it's better to just build without a Python interpreter.nix.conf? (See Nix manual)sandbox = relaxedsandbox = truenix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)Add a 👍 reaction to pull requests you find important.