Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overhaul GH Action workflows #440

Merged
merged 7 commits into from
Mar 5, 2024
Merged

Overhaul GH Action workflows #440

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
ac83b7c
fetch-and-ingest: Add `trial_name` input
joverlee521 Feb 27, 2024
94fffb4
fetch-and-ingest: Add `fetch_from_database` input
joverlee521 Feb 28, 2024
25c00e4
Use `--env` option of `nextstrain build`
joverlee521 Feb 28, 2024
fca290f
fetch-and-ingest: refactor to use pathogen-repo-build
joverlee521 Feb 28, 2024
8fe1390
fetch-and-ingest: Add `image` input
joverlee521 Feb 28, 2024
bae20ea
fetch-and-ingest: Use `TEST_SLACK_CHANNEL` for trials
joverlee521 Feb 28, 2024
3d92f48
fetch-and-ingest: fix `FETCH_FROM_DATABASE` variable
joverlee521 Feb 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/fetch-and-ingest-genbank-master.yml
View file
Open in desktop
jameshadfield marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it'd be possible to combine the gisaid + genbank workflows into one (by adding a bit more conditional logic) - the workflow files are almost identical. Did you choose not to do this to allow the actions page to easily distinguish between the two?

That brings up a related point -- if we are using the actions page to monitor progress for the foreseeable future, is the best view into this to filter by event:scheduled to ignore trial / branch runs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you choose not to do this to allow the actions page to easily distinguish between the two?

Yeah, I think it's nice to be able to easily distinguish between GISAID and GenBank workflows. (The pathogen status page currently groups by repo/workflow name). Also, the scheduled GitHub Action workflows do not use inputs, so I think the conditionals to trigger scheduled workflows with different default values could get pretty messy.

if we are using the actions page to monitor progress for the foreseeable future, is the best view into this to filter by event:scheduled to ignore trial / branch runs?

Yup, that's should be the easiest way to monitor the automated runs for now. I think it'd also be reasonable to add a new status page that only filters to the scheduled events.

Original file line number Diff line number Diff line change
Expand Up @@ -60,11 +60,11 @@ jobs:
name: Set config overrides
env:
TRIAL_NAME: ${{ inputs.trial_name }}
FETCH_FROM_DATABASE: ${{ inputs.fetch_from_database != '' && inputs.fetch_from_database || true }}
FETCH_FROM_DATABASE: ${{ github.event_name != 'workflow_dispatch' && true || inputs.fetch_from_database }}
run: |
config="--config"

if [[ "$FETCH_FROM_DATABASE" ]]; then
if "$FETCH_FROM_DATABASE"; then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This relies on running the programs true and false... 🫣

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for flagging! Updating in #444

config+=" fetch_from_database=True"
else
config+=" fetch_from_database=False"
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/fetch-and-ingest-gisaid-master.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,11 +59,11 @@ jobs:
name: Set config overrides
env:
TRIAL_NAME: ${{ inputs.trial_name }}
FETCH_FROM_DATABASE: ${{ inputs.fetch_from_database != '' && inputs.fetch_from_database || true }}
FETCH_FROM_DATABASE: ${{ github.event_name != 'workflow_dispatch' && true || inputs.fetch_from_database }}
run: |
config="--config"

if [[ "$FETCH_FROM_DATABASE" ]]; then
if "$FETCH_FROM_DATABASE"; then
config+=" fetch_from_database=True"
else
config+=" fetch_from_database=False"
Expand Down