Add pip_extra_args #3081

amitani · 2025-01-23T19:23:10Z

Tracking issue

Why are the changes needed?

We have a few path dependencies in pyproject.toml and poetry.lock, and only want to install packages from external sources, which requires adding --no-directory to poetry install. This is a standard practice when installing dependencies without code.
See
https://python-poetry.org/docs/faq/#poetry-busts-my-docker-cache-because-it-requires-me-to-copy-my-source-files-in-before-installing-3rd-party-dependencies

What changes were proposed in this pull request?

pip_extra_args is a specifiable parameter for ImageSpec, and appended when running poetry install, uv sync and pip install.

How was this patch tested?

I tested locally with a poetry project with internal path dependency by running pyflyte build. Planning to add unit tests once the implementation get LGTM.

Setup process

Screenshots

Check all the applicable boxes

I updated the documentation accordingly.
All new and existing tests passed.
All commits are signed-off.

Related PRs

#3025

Docs link

Summary by Bito

Added new pip_extra_args parameter to ImageSpec class for enhanced package installation flexibility, enabling custom pip installation arguments for path dependencies in poetry projects. Enhanced test suite implementation to validate pip_extra_args parameter, including test cases for --no-install-package flag and path dependency handling with --no-directory option. Tests ensure proper integration with both poetry and pip installation workflows.

Unit tests added: True

Estimated effort to review (1-5, lower is better): 1

welcome · 2025-01-23T19:23:13Z

Thank you for opening this pull request! 🙌

These tips will help get your PR across the finish line:

Most of the repos have a PR template; if not, fill it out to the best of your knowledge.
Sign off your commits (Reference: DCO Guide).

flyte-bot · 2025-01-23T19:23:26Z

Code Review Agent Run #bf76cd

Actionable Suggestions - 1

flytekit/image_spec/image_spec.py - 1
- Missing class attribute for docstring · Line 51-51

Review Details

Files reviewed - 2 · Commit Range: 2569f80..2569f80
- flytekit/image_spec/default_builder.py
- flytekit/image_spec/image_spec.py
Files skipped - 0
Tools
- Whispers (Secret Scanner) - ✔︎ Successful
- Detect-secrets (Secret Scanner) - ✔︎ Successful
- MyPy (Static Code Analysis) - ✔︎ Successful
- Astral Ruff (Static Code Analysis) - ✔︎ Successful

AI Code Review powered by

flyte-bot · 2025-01-23T19:26:07Z

Changelist by Bito

This pull request implements the following key changes.

Key Change	Files Impacted
Feature Improvement - Enhanced Package Installation Configuration	- `default_builder.py` - Added support for pip_extra_args in python package installation - `image_spec.py` - Added pip_extra_args parameter to ImageSpec class - `test_default_builder.py` - Added test cases for pip_extra_args functionality

flytekit/image_spec/image_spec.py

thomasjpfan

Can you update a list to make sure that the new argument is used?

flytekit/image_spec/image_spec.py

flyte-bot · 2025-01-23T22:35:29Z

Code Review Agent Run #110b04

Actionable Suggestions - 1

flytekit/image_spec/default_builder.py - 1
- Potential issue with pip_extra_args handling · Line 220-220

Review Details

Files reviewed - 2 · Commit Range: 4281029..4281029
- flytekit/image_spec/default_builder.py
- flytekit/image_spec/image_spec.py
Files skipped - 0
Tools
- Whispers (Secret Scanner) - ✔︎ Successful
- Detect-secrets (Secret Scanner) - ✔︎ Successful
- MyPy (Static Code Analysis) - ✔︎ Successful
- Astral Ruff (Static Code Analysis) - ✔︎ Successful

AI Code Review powered by

flytekit/image_spec/default_builder.py

Signed-off-by: amitani <[email protected]>

flyte-bot · 2025-01-24T20:47:23Z

Code Review Agent Run #581024

Actionable Suggestions - 0

Additional Suggestions - 1

flytekit/image_spec/image_spec.py - 1
- Consider validating pip extra args input · Line 86-86

Review Details

Files reviewed - 2 · Commit Range: 31a8f47..31a8f47
- flytekit/image_spec/default_builder.py
- flytekit/image_spec/image_spec.py
Files skipped - 0
Tools
- Whispers (Secret Scanner) - ✔︎ Successful
- Detect-secrets (Secret Scanner) - ✔︎ Successful
- MyPy (Static Code Analysis) - ✔︎ Successful
- Astral Ruff (Static Code Analysis) - ✔︎ Successful

AI Code Review powered by

codecov · 2025-01-24T21:18:56Z

Codecov Report

Attention: Patch coverage is 33.33333% with 2 lines in your changes missing coverage. Please review.

Project coverage is 79.66%. Comparing base (06db981) to head (31a8f47).
Report is 2 commits behind head on master.

Files with missing lines	Patch %	Lines
flytekit/image_spec/default_builder.py	0.00%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3081      +/-   ##
==========================================
+ Coverage   74.82%   79.66%   +4.83%     
==========================================
  Files         202      202              
  Lines       21446    21461      +15     
  Branches     2763     2766       +3     
==========================================
+ Hits        16048    17097    +1049     
+ Misses       4626     3595    -1031     
+ Partials      772      769       -3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: amitani <[email protected]>

flyte-bot · 2025-01-24T23:27:24Z

Code Review Agent Run #c147c6

Actionable Suggestions - 0

Review Details

Files reviewed - 1 · Commit Range: 31a8f47..70d4f36
- tests/flytekit/unit/core/image_spec/test_default_builder.py
Files skipped - 0
Tools
- Whispers (Secret Scanner) - ✔︎ Successful
- Detect-secrets (Secret Scanner) - ✔︎ Successful
- MyPy (Static Code Analysis) - ✔︎ Successful
- Astral Ruff (Static Code Analysis) - ✔︎ Successful

AI Code Review powered by

welcome · 2025-01-25T01:03:16Z

Congrats on merging your first pull request! 🎉

Signed-off-by: amitani <[email protected]> Signed-off-by: lu00122 <[email protected]>

amitani requested review from wild-endeavor, kumare3, eapolinario, pingsutw, cosmicBboy, samhita-alla, thomasjpfan and Future-Outlier as code owners January 23, 2025 19:23

flyte-bot reviewed Jan 23, 2025

View reviewed changes

flytekit/image_spec/image_spec.py Outdated Show resolved Hide resolved

thomasjpfan reviewed Jan 23, 2025

View reviewed changes

flytekit/image_spec/image_spec.py Outdated Show resolved Hide resolved

amitani force-pushed the pip_extra_args branch from 5a2c44f to 4281029 Compare January 23, 2025 21:42

flyte-bot reviewed Jan 23, 2025

View reviewed changes

flytekit/image_spec/default_builder.py Show resolved Hide resolved

wild-endeavor previously approved these changes Jan 24, 2025

View reviewed changes

amitani dismissed wild-endeavor’s stale review via 3757b78 January 24, 2025 19:40

amitani force-pushed the pip_extra_args branch from 3757b78 to f65bc86 Compare January 24, 2025 20:36

Add pip_extra_args

31a8f47

Signed-off-by: amitani <[email protected]>

amitani force-pushed the pip_extra_args branch from f65bc86 to 31a8f47 Compare January 24, 2025 20:44

Add test

70d4f36

Signed-off-by: amitani <[email protected]>

thomasjpfan approved these changes Jan 24, 2025

View reviewed changes

wild-endeavor merged commit 5be0a6c into flyteorg:master Jan 25, 2025
103 of 106 checks passed

amitani deleted the pip_extra_args branch January 26, 2025 22:02

ChihTsungLu pushed a commit to ChihTsungLu/flytekit that referenced this pull request Jan 27, 2025

Add pip_extra_args (flyteorg#3081)

20d8c01

Signed-off-by: amitani <[email protected]> Signed-off-by: lu00122 <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add pip_extra_args #3081

Add pip_extra_args #3081

amitani commented Jan 23, 2025 •

edited by flyte-bot

Loading

welcome bot commented Jan 23, 2025

flyte-bot commented Jan 23, 2025 •

edited

Loading

Code Review Agent Run #bf76cd

flyte-bot commented Jan 23, 2025 •

edited

Loading

Changelist by Bito

thomasjpfan left a comment

flyte-bot commented Jan 23, 2025 •

edited

Loading

Code Review Agent Run #110b04

flyte-bot commented Jan 24, 2025 •

edited

Loading

Code Review Agent Run #581024

codecov bot commented Jan 24, 2025 •

edited

Loading

flyte-bot commented Jan 24, 2025 •

edited

Loading

Code Review Agent Run #c147c6

welcome bot commented Jan 25, 2025

Add pip_extra_args #3081

Add pip_extra_args #3081

Conversation

amitani commented Jan 23, 2025 • edited by flyte-bot Loading

Tracking issue

Why are the changes needed?

What changes were proposed in this pull request?

How was this patch tested?

Setup process

Screenshots

Check all the applicable boxes

Related PRs

Docs link

Summary by Bito

welcome bot commented Jan 23, 2025

flyte-bot commented Jan 23, 2025 • edited Loading

Code Review Agent Run #bf76cd

flyte-bot commented Jan 23, 2025 • edited Loading

Changelist by Bito

thomasjpfan left a comment

Choose a reason for hiding this comment

flyte-bot commented Jan 23, 2025 • edited Loading

Code Review Agent Run #110b04

flyte-bot commented Jan 24, 2025 • edited Loading

Code Review Agent Run #581024

codecov bot commented Jan 24, 2025 • edited Loading

Codecov Report

flyte-bot commented Jan 24, 2025 • edited Loading

Code Review Agent Run #c147c6

welcome bot commented Jan 25, 2025

amitani commented Jan 23, 2025 •

edited by flyte-bot

Loading

flyte-bot commented Jan 23, 2025 •

edited

Loading

flyte-bot commented Jan 23, 2025 •

edited

Loading

flyte-bot commented Jan 23, 2025 •

edited

Loading

flyte-bot commented Jan 24, 2025 •

edited

Loading

codecov bot commented Jan 24, 2025 •

edited

Loading

flyte-bot commented Jan 24, 2025 •

edited

Loading