Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace pip and pip-tools with uv, update ruff #3324

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 7 additions & 10 deletions .github/workflows/backend.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,24 +35,21 @@ jobs:
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
- name: Set up uv
uses: astral-sh/setup-uv@v2
with:
python-version: '3.11'
cache: pip
cache-dependency-path: |
requirements/default.txt
requirements/dev.txt
requirements/lint.txt
requirements/test.txt
enable-cache: true
- name: Install Python
run: uv python install 3.11
- run: >
pip install
uv pip install
-r requirements/default.txt
-r requirements/dev.txt
-r requirements/test.txt
-r requirements/lint.txt

# Set environment variables
- run: pip install pytest-dotenv
- run: uv pip install pytest-dotenv
- run: >
sed
-e 's#^DATABASE_URL=.*#DATABASE_URL=postgres://pontoon:pontoon@localhost/pontoon#'
Expand Down
11 changes: 7 additions & 4 deletions .github/workflows/py-lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,16 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
- name: Set up uv
uses: astral-sh/setup-uv@v2
with:
python-version: '3.11'
enable-cache: true
- name: Install Python
run: uv python install 3.11
- name: Install Dependencies
run: |
pip install -U pip
pip install -r requirements/lint.txt
uv pip install -U pip
uv pip install -r requirements/lint.txt
- name: ruff lint
run: ruff check pontoon
- name: ruff format
Expand Down
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
*.py[co]
*.sw[po]
/pontoon/coverage.xml
pip-log.txt
docs/_gh-pages
build.py
build
Expand Down
20 changes: 11 additions & 9 deletions CONTRIBUTING.rst
Original file line number Diff line number Diff line change
Expand Up @@ -225,10 +225,12 @@ Direct dependencies for Pontoon are distributed across four files:
3. ``requirements/test.in``: Testing
4. ``requirements/lint.in``: Linting

In order to pin and hash the direct and indirect dependencies, we use `pip-compile <https://pypi.org/project/pip-tools/>`_,
which yields corresponding ``*.txt`` files. These ``*.txt`` files contain all direct and indirect dependencies,
and can be used for installation with ``pip``. After any change to the ``*.in`` files,
you should run the following command to update all ``requirements/*.txt`` files.
In order to pin and hash the direct and indirect dependencies, we use
`uv pip compile <https://docs.astral.sh/uv/#the-pip-interface>`_, which yields
corresponding ``*.txt`` files. These ``*.txt`` files contain all direct and
indirect dependencies, and can be used for installation with ``uv pip``. After any
change to the ``*.in`` files, you should run the following command to update all
``requirements/*.txt`` files.

.. code-block:: shell

Expand All @@ -247,7 +249,7 @@ Once you are done adding, removing or updating requirements, rebuild your docker
If there are problems, it'll tell you.

To upgrade existing dependencies within the given constraints of the input
files, you can pass options through to the ``pip-compile`` invocations, i.e.
files, you can pass options through to the ``uv pip compile`` invocations, i.e.

.. code-block:: shell

Expand All @@ -265,9 +267,9 @@ a virtualenv to build docs, do this:
.. code-block:: shell

$ cd docs/
$ virtualenv venv
$ source venv/bin/activate
$ pip install --require-hashes -r requirements.txt
$ uv venv
$ source .venv/bin/activate
$ uv pip install -r requirements.txt

Then, to build the docs, run this:

Expand Down Expand Up @@ -378,7 +380,7 @@ steps, as they don't affect your setup if nothing has changed:
git pull origin main

# Install new dependencies or update existing ones.
pip install -U --force --require-hashes -r requirements/default.txt
uv pip install -U --force -r requirements/default.txt

# Run database migrations.
python manage.py migrate
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ help:
@echo " dumpdb Create a postgres database dump with timestamp used as file name"
@echo " loaddb Load a database dump into postgres, file name in DB_DUMP_FILE"
@echo " sync-projects Runs the synchronization task on all projects"
@echo " requirements Compiles all requirements files with pip-compile\n"
@echo " requirements Compiles all requirements files with uv pip compile\n"

translate/dist:
make build-translate
Expand Down Expand Up @@ -143,5 +143,5 @@ sync-projects:

requirements:
# Pass --upgrade to upgrade all dependencies
# The arguments are passed through to pip-compile
# The arguments are passed through to `uv pip compile`
"${DC}" run --rm server //app/docker/compile_requirements.sh ${opts}
2 changes: 1 addition & 1 deletion bin/post_compile
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
# Install requirements/dev.txt on staging
if [ "$DJANGO_STAGE" = True ]; then
echo "Installing dev requirements..."
pip install --require-hashes -r requirements/dev.txt
uv pip install requirements/dev.txt
fi

# Compile static assets
Expand Down
4 changes: 2 additions & 2 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,8 @@ WORKDIR /app

# Install Pontoon Python requirements
COPY requirements/* /app/requirements/
RUN pip install -U 'pip>=8' && \
pip install --no-cache-dir --require-hashes -r requirements/default.txt -r requirements/dev.txt -r requirements/test.txt -r requirements/lint.txt
RUN uv pip install -U 'pip>=8' && \
uv pip install --no-cache-dir --require-hashes -r requirements/default.txt -r requirements/dev.txt -r requirements/test.txt -r requirements/lint.txt

# Create the app user
RUN groupadd -r --gid=${GROUP_ID} pontoon && useradd --uid=${USER_ID} --no-log-init -r -m -g pontoon pontoon
Expand Down
10 changes: 5 additions & 5 deletions docker/compile_requirements.sh
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
#!/bin/bash

# This compiles all requirements files with pip-compile.
# This compiles all requirements files with uv pip compile.
# You should always use this script, because dev.txt and test.txt depend on default.txt.

export CUSTOM_COMPILE_COMMAND="./docker/compile_requirements.sh"

pip-compile --generate-hashes --resolver=backtracking $@ requirements/default.in
pip-compile --generate-hashes --resolver=backtracking $@ requirements/dev.in
pip-compile --generate-hashes --resolver=backtracking $@ requirements/lint.in
pip-compile --generate-hashes --resolver=backtracking $@ requirements/test.in
uv pip compile --generate-hashes --resolver=backtracking $@ requirements/default.in
uv pip compile --generate-hashes --resolver=backtracking $@ requirements/dev.in
uv pip compile --generate-hashes --resolver=backtracking $@ requirements/lint.in
uv pip compile --generate-hashes --resolver=backtracking $@ requirements/test.in
1 change: 0 additions & 1 deletion requirements/dev.in
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,3 @@
django-debug-toolbar==4.3.0
django-extensions==3.2.3
django-sslserver==0.22
pip-tools==7.4.1
34 changes: 0 additions & 34 deletions requirements/dev.txt
Original file line number Diff line number Diff line change
Expand Up @@ -10,16 +10,6 @@ asgiref==3.8.1 \
# via
# -c requirements/default.txt
# django
build==1.2.1 \
--hash=sha256:526263f4870c26f26c433545579475377b2b7588b6f1eac76a001e873ae3e19d \
--hash=sha256:75e10f767a433d9a86e50d83f418e83efc18ede923ee5ff7df93b6cb0306c5d4
# via pip-tools
click==8.1.7 \
--hash=sha256:ae74fb96c20a0277a1d615f1e4d73c8414f5a98db8b799a7931d1582f3390c28 \
--hash=sha256:ca9853ad459e787e2192211578cc907e7594e294c7ccc834310722b41b9ca6de
# via
# -c requirements/default.txt
# pip-tools
django==4.2.11 \
--hash=sha256:6e6ff3db2d8dd0c986b4eec8554c8e4f919b5c1ff62a5b4390c17aff2ed6e5c4 \
--hash=sha256:ddc24a0a8280a0430baa37aff11f28574720af05888c62b7cfe71d219f4599d3
Expand All @@ -39,34 +29,10 @@ django-extensions==3.2.3 \
django-sslserver==0.22 \
--hash=sha256:c598a363d2ccdc2421c08ddb3d8b0973f80e8e47a3a5b74e4a2896f21c2947c5
# via -r requirements/dev.in
packaging==24.0 \
--hash=sha256:2ddfb553fdf02fb784c234c7ba6ccc288296ceabec964ad2eae3777778130bc5 \
--hash=sha256:eb82c5e3e56209074766e6885bb04b8c38a0c015d0a30036ebe7ece34c9989e9
# via build
pip-tools==7.4.1 \
--hash=sha256:4c690e5fbae2f21e87843e89c26191f0d9454f362d8acdbd695716493ec8b3a9 \
--hash=sha256:864826f5073864450e24dbeeb85ce3920cdfb09848a3d69ebf537b521f14bcc9
# via -r requirements/dev.in
pyproject-hooks==1.0.0 \
--hash=sha256:283c11acd6b928d2f6a7c73fa0d01cb2bdc5f07c57a2eeb6e83d5e56b97976f8 \
--hash=sha256:f271b298b97f5955d53fb12b72c1fb1948c22c1a6b70b315c54cedaca0264ef5
# via
# build
# pip-tools
sqlparse==0.4.4 \
--hash=sha256:5430a4fe2ac7d0f93e66f1efc6e1338a41884b7ddf2a350cedd20ccc4d9d28f3 \
--hash=sha256:d446183e84b8349fa3061f0fe7f06ca94ba65b426946ffebe6e3e8295332420c
# via
# -c requirements/default.txt
# django
# django-debug-toolbar
wheel==0.43.0 \
--hash=sha256:465ef92c69fa5c5da2d1cf8ac40559a8c940886afcef87dcf14b9470862f1d85 \
--hash=sha256:55c570405f142630c6b9f72fe09d9b67cf1477fcf543ae5b8dcb1f5b7377da81
# via pip-tools

# WARNING: The following packages were not pinned, but pip requires them to be
# pinned when the requirements file includes hashes and the requirement is not
# satisfied by a package already installed. Consider using the --allow-unsafe flag.
# pip
# setuptools
2 changes: 1 addition & 1 deletion requirements/lint.in
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,4 @@
# The dependencies are sorted by alphabetical order.
# Dependencies that do not come from pypi (eg. eggs from github) are listed at the end of the list.
# -------------------------------------------------------------------------------------------------
ruff==0.4.6
ruff==0.6.4
37 changes: 19 additions & 18 deletions requirements/lint.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,23 @@
#
# ./docker/compile_requirements.sh
#
ruff==0.4.6 \
--hash=sha256:04a80acfc862e0e1630c8b738e70dcca03f350bad9e106968a8108379e12b31f \
--hash=sha256:0cf5cc02d3ae52dfb0c8a946eb7a1d6ffe4d91846ffc8ce388baa8f627e3bd50 \
--hash=sha256:1fa8561489fadf483ffbb091ea94b9c39a00ed63efacd426aae2f197a45e67fc \
--hash=sha256:1ff930d6e05f444090a0139e4e13e1e2e1f02bd51bb4547734823c760c621e79 \
--hash=sha256:3a6a0a4f4b5f54fff7c860010ab3dd81425445e37d35701a965c0248819dde7a \
--hash=sha256:3f9ced5cbb7510fd7525448eeb204e0a22cabb6e99a3cb160272262817d49786 \
--hash=sha256:4d5b914818d8047270308fe3e85d9d7f4a31ec86c6475c9f418fbd1624d198e0 \
--hash=sha256:4f02284335c766678778475e7698b7ab83abaf2f9ff0554a07b6f28df3b5c259 \
--hash=sha256:602ebd7ad909eab6e7da65d3c091547781bb06f5f826974a53dbe563d357e53c \
--hash=sha256:735a16407a1a8f58e4c5b913ad6102722e80b562dd17acb88887685ff6f20cf6 \
--hash=sha256:9018bf59b3aa8ad4fba2b1dc0299a6e4e60a4c3bc62bbeaea222679865453062 \
--hash=sha256:a769ae07ac74ff1a019d6bd529426427c3e30d75bdf1e08bb3d46ac8f417326a \
--hash=sha256:a797a87da50603f71e6d0765282098245aca6e3b94b7c17473115167d8dfb0b7 \
--hash=sha256:be47700ecb004dfa3fd4dcdddf7322d4e632de3c06cd05329d69c45c0280e618 \
--hash=sha256:ea3424793c29906407e3cf417f28fc33f689dacbbadfb52b7e9a809dd535dcef \
--hash=sha256:ef995583a038cd4a7edf1422c9e19118e2511b8ba0b015861b4abd26ec5367c5 \
--hash=sha256:f13410aabd3b5776f9c5699f42b37a3a348d65498c4310589bc6e5c548dc8a2f
ruff==0.6.4 \
--hash=sha256:0308610470fcc82969082fc83c76c0d362f562e2f0cdab0586516f03a4e06ec6 \
--hash=sha256:0b52387d3289ccd227b62102c24714ed75fbba0b16ecc69a923a37e3b5e0aaaa \
--hash=sha256:0ea086601b22dc5e7693a78f3fcfc460cceabfdf3bdc36dc898792aba48fbad6 \
--hash=sha256:34d5efad480193c046c86608dbba2bccdc1c5fd11950fb271f8086e0c763a5d1 \
--hash=sha256:50e30b437cebef547bd5c3edf9ce81343e5dd7c737cb36ccb4fe83573f3d392e \
--hash=sha256:549daccee5227282289390b0222d0fbee0275d1db6d514550d65420053021a58 \
--hash=sha256:66dbfea86b663baab8fcae56c59f190caba9398df1488164e2df53e216248baa \
--hash=sha256:7862f42fc1a4aca1ea3ffe8a11f67819d183a5693b228f0bb3a531f5e40336fc \
--hash=sha256:803b96dea21795a6c9d5bfa9e96127cc9c31a1987802ca68f35e5c95aed3fc0d \
--hash=sha256:932063a03bac394866683e15710c25b8690ccdca1cf192b9a98260332ca93408 \
--hash=sha256:ac3b5bfbee99973f80aa1b7cbd1c9cbce200883bdd067300c22a6cc1c7fba212 \
--hash=sha256:ac4b75e898ed189b3708c9ab3fc70b79a433219e1e87193b4f2b77251d058d14 \
--hash=sha256:bedff9e4f004dad5f7f76a9d39c4ca98af526c9b1695068198b3bda8c085ef60 \
--hash=sha256:c44536df7b93a587de690e124b89bd47306fddd59398a0fb12afd6133c7b3818 \
--hash=sha256:c4b153fc152af51855458e79e835fb6b933032921756cec9af7d0ba2aa01a258 \
--hash=sha256:d02a4127a86de23002e694d7ff19f905c51e338c72d8e09b56bfb60e1681724f \
--hash=sha256:eebe4ff1967c838a1a9618a5a59a3b0a00406f8d7eefee97c70411fefc353617 \
--hash=sha256:f0f8968feea5ce3777c0d8365653d5e91c40c31a81d95824ba61d871a11b8523
# via -r requirements/lint.in