Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trigger workflows more selectively #355

Merged
merged 8 commits into from
Nov 12, 2024
Merged

Conversation

scotts
Copy link
Contributor

@scotts scotts commented Nov 9, 2024

I'm trying to make our workflow triggering smarter. For example, when we're making doc changes, we're still triggering Mac and CUDA wheels, and those jobs take a long time and we just ignore them anyway. I'm less concerned about wasting resources, and more concerned about making sure we see important signals. All workflows have a self reference. In addition to that, my reasoning on why triggering what when:

build_ffmpeg.yaml
Only triggered when the script itself is updated. The script has no dependence on our code.

cpp_tests.yaml
Triggered when anything under src/torchcodec/* or test/* is changed. That means Python only changes will also trigger these tests, but I think trying to differentiate at that level is going to be too subtle and error prone. Doc changes do not trigger the C++ tests.

docs.yaml
Triggered when anything under src/torchcodec/* or docs/* is changed. Our source code has docs, and obviously we build the docs when docs files change. Notably we don't need to trigger the docs when tests change.

linux_wheel.yaml, linux_cuda_wheel.yaml, macos_wheel.yaml
All are triggered when anything under src/torchcodec/*, test/* or packaging/* is changed. These are our main test drivers, so we want these to run when all source code, tests or packaging scripts change. Doc changes do not need to trigger wheel builds and tests.

reference_resources.yaml
Only triggered when the script itself is updated. The script has no dependence on our code.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Meta Open Source bot. label Nov 9, 2024
@scotts scotts marked this pull request as ready for review November 9, 2024 03:18
@scotts scotts force-pushed the workflow_partition branch from 906dbf2 to 17a3ff5 Compare November 9, 2024 18:34
Copy link
Member

@NicolasHug NicolasHug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @scotts , this looks good, happy to try this out. The best way to test this probably is to merge the PR and pay extra attention to the CI signals within the next few days.

Doc changes do not need to trigger wheel builds

Note that this may soon be the case, depending on how we go with #335, but this PR can be merged first nonetheless.

.github/workflows/linux_cuda_wheel.yaml Show resolved Hide resolved
@scotts scotts merged commit 10be66b into pytorch:main Nov 12, 2024
56 of 59 checks passed
@scotts scotts deleted the workflow_partition branch November 12, 2024 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Meta Open Source bot.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants