Skip to content

Commit

Permalink
Add recce cloud compatible output
Browse files Browse the repository at this point in the history
Signed-off-by: popcorny <[email protected]>
  • Loading branch information
popcornylu committed Jun 5, 2024
1 parent 39558ad commit 28d9e38
Showing 1 changed file with 7 additions and 21 deletions.
28 changes: 7 additions & 21 deletions .github/workflows/recce_ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,30 +22,16 @@ jobs:
pip install -r requirements.txt
- name: Install Recce
run: |
pip install recce-nightly
- name: Add packages.yml file
run: |
echo '${{ vars.PACKAGES_YAML }}' > packages.yml
pip install recce
- name: Prepare dbt Base environment
run: |
if aws s3 cp s3://$AWS_S3_BUCKET/${{ github.event.pull_request.base.sha }}.tar.gz .; then
echo "Base environment found in S3"
tar -xvf ${{ github.event.pull_request.base.sha }}.tar.gz
else
echo "Base environment not found in S3. Running dbt to create base environment"
git checkout ${{ github.event.pull_request.base.sha }}
dbt deps
dbt seed --target ${{ env.DBT_BASE_TARGET }} --target-path target-base
dbt run --target ${{ env.DBT_BASE_TARGET }} --target-path target-base
dbt docs generate --target ${{ env.DBT_BASE_TARGET }} --target-path target-base
fi
gh repo set-default ${{ github.repository }}
run_id=$(gh run list --workflow ${WORKFLOW_BASE} --status success --limit 1 --json databaseId --jq '.[0].databaseId')
gh run download $run_id -n target -D target-base
gh run download $run_id -n duckdb
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_REGION: ${{ secrets.AWS_REGION }}
AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}
DBT_BASE_TARGET: "prod"

GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
WORKFLOW_BASE: ".github/workflows/dbt_base.yml"
- name: Prepare dbt Current environment
run: |
git checkout ${{ github.event.pull_request.head.sha }}
Expand Down

0 comments on commit 28d9e38

Please sign in to comment.