Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

automate internal release workflow. #1276

Closed
wants to merge 11 commits into from
Closed

automate internal release workflow. #1276

Changes from 3 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
228d7b8
draft initial additon to internal release workflow for cron implement…
McKnight-42 Jul 10, 2024
3f50932
minor update, to grab from last successful run of workflow not last r…
McKnight-42 Jul 10, 2024
2d1c9a4
try to target specific release-internal.yml file to check for commit …
McKnight-42 Jul 10, 2024
7c82ab8
update workflow
McKnight-42 Jul 11, 2024
ad4d43b
typo in file name
McKnight-42 Jul 11, 2024
d2fd0b6
Merge branch 'main' of https://github.com/dbt-labs/dbt-bigquery into …
McKnight-42 Jul 19, 2024
0c88154
update workflow based on feedback
McKnight-42 Jul 19, 2024
ff10395
add validation for None value
McKnight-42 Jul 19, 2024
375509b
feddback from friday to update workflow and implementent dry_run
McKnight-42 Jul 22, 2024
1f176ee
Merge branch 'main' of https://github.com/dbt-labs/dbt-bigquery into …
McKnight-42 Jul 22, 2024
450e9a7
update fetch reach to try and make sure we can check everything
McKnight-42 Jul 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 30 additions & 4 deletions .github/workflows/release-internal.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
#
# When?
#
# Manual trigger.
# Manual trigger or cron job every wednesday morning.
McKnight-42 marked this conversation as resolved.
Show resolved Hide resolved

name: "Release to Cloud"
run-name: "Release to Cloud off of ${{ inputs.ref }}"
Expand All @@ -32,20 +32,46 @@ on:
required: true
default: false

schedule:
- cron: '0 13 * * 3' # Every Wednesday at 8:00 AM central time

defaults:
run:
shell: bash

jobs:
get-last-successful-release-run:
runs-on: ubuntu-latest
outputs:
commit_sha: ${{ steps.get_last_successful_release_run.outputs.commit_sha }}
steps:
- name: Get last successful Release to Cloud run details
id: get_last_successful_release_run
run: |
last_run_url=$(curl --silent -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" "https://api.github.com/repos/${{ github.repository }}/actions/runs?event=workflow_dispatch&status=success&branch=${{ github.ref }}&workflow_file=release.internal.yml" | jq -r '.workflow_runs[0].url')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is pretty clever

commit_sha=$(curl --silent -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" $last_run_url | jq -r '.head_sha')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way this can be None? Might be nice to validate and exit here if it's empty.

echo "::set-output name=commit_sha::$commit_sha"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs to be the new syntax that uses >> GITHUB_OUTPUT. I forget what it is off the top of my head, but it's something like echo "set commit_sha=$commit_sha" >> GITHUB_OUTPUT.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Helpful docs! https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#example-masking-a-generated-output-within-a-single-job


check-for-new-commits:
runs-on: ubuntu-latest
needs: get-last-successful-release-run
outputs:
new_commits: ${{ steps.check_new_commits.outputs.new_commits }}
steps:
- name: Check for new commits since last release
id: check_new_commits
run: |
last_release_date=$(curl --silent "https://api.github.com/repos/${{ github.repository }}/releases/tags/${{ needs.get-last-successful-run.outputs.commit_sha }}" | jq -r .published_at)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we want to retrieve commits not releases (i.e. we may not have a github release yet). Looks like this could work:
https://api.github.com/repos/dbt-labs/dbt-bigquery/commits/2c77893d0fb7b5d76ba85cc16712b9563a60329b

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updating this more this morning i'm starting to wonder if this should be its own action that triggers our current workflow to supply the fields we need that way. thoughts?

new_commits=$(git rev-list --count --since="$last_release_date" ${{ inputs.ref }})
echo "::set-output name=new_commits::$new_commits"

invoke-reusable-workflow:
if: needs.check-for-new-commits.output.new_commits != '0'
name: "Build and Release Internally"

uses: "dbt-labs/dbt-release/.github/workflows/internal-archive-release.yml@main"

with:
package_test_command: "${{ inputs.package_test_command }}"
dbms_name: "bigquery"
ref: "${{ inputs.ref }}"
skip_tests: "${{ inputs.skip_tests }}"

secrets: "inherit"
McKnight-42 marked this conversation as resolved.
Show resolved Hide resolved
Loading