Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove alpha specs on non-RAPIDS dependencies #144

Open
jameslamb opened this issue Jan 27, 2025 · 0 comments
Open

Remove alpha specs on non-RAPIDS dependencies #144

jameslamb opened this issue Jan 27, 2025 · 0 comments

Comments

@jameslamb
Copy link
Member

jameslamb commented Jan 27, 2025

Description

pip 25.0 was recently released (changelog). It includes this change:

The inclusion of packaging 24.2 changes how pre-release specifiers with < and > behave. Including a pre-release version with these specifiers now implies accepting pre-releases (e.g., <2.0dev can include 1.0rc1). To avoid implying pre-releases, avoid specifying them (e.g., use <2.0). The exception is !=, which never implies pre-releases. (#13163)

Relevant packaging change: pypa/packaging#794

This slight behavior change could result in some changed pip solves that affect RAPIDS.

For example, consider:

docker run \
  --rm \
  -it python:3.12 \
  /bin/bash -c 'pip install "pip<25"; pip install --dry-run "cython>=0.29.8,<3.0.0rc1"'
  • pip <25.0 finds cython==0.29.37
  • pip >=25.0 finds the pre-release cython==3.0.0b3

The presence of <...rc1 tells pip "pre-releases are OK, even if --pre was not passed to pip install", and then 3.0.0b3 satisfies this particular constraint because of the rules described in PEP 404 (link)):

Within a numeric release (1.0, 2.7.3), the following suffixes are permitted and MUST be ordered as shown:
.devN, aN, bN, rcN, <no suffix>, .postN

RAPIDS projects currently use such constraints in situations where it's not actually desirable to get pre-releases. For example, cudf 25.02 has these runtime dependencies:

numpy>=1.23,<3.0a0
pandas>=2.0,<2.2.4dev0

(cudf/dependencies.yaml)

With pip>25.0, these would tell pip that pre-releases of numpy or pandas are acceptable, which is not really the intention and could have negative side-effects (for example, regular releases tend to be more reliable as they've gone through more extensive testing).

We should audit the dependency constraints across RAPIDS and remove modifies like dev, a, b, and rc wherever allowing pre-releases is undesirable.

Benefits of this work

  • reduced risk of issues caused by unintentionally picking up pre-releases
  • fasts pip solves
  • avoids surprises for users (e.g. no numpy pre-releases sneaking into environments transitively via a cudf dependency)

Acceptance Criteria

  • RAPIDS projects only use pre-release specifiers for dependencies where they explicitly want to allow pre-releases

Approach

@bdice @vyasr and I talked about this offline and agreed the right fix is probably to change specs like this:

# before
pandas>=2.0,<2.2.4dev0

# after
pandas>=2.0,<2.2.4

Where possible, use rapids-reviser to update these specs.

Notes

N/A

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant