Skip to content
This repository has been archived by the owner on Jun 4, 2024. It is now read-only.

Migrated Drone pipelines to GHA #983

Merged
merged 14 commits into from
Feb 17, 2024
Merged

Migrated Drone pipelines to GHA #983

merged 14 commits into from
Feb 17, 2024

Conversation

fheinecke
Copy link
Contributor

@fheinecke fheinecke commented Dec 13, 2023

This PR migrates all currently used Drone pipelines for this repo to GHA.

This pull request is pretty large. I had originally planned to split it into several, however after getting started on this project I decided that it would take significantly more effort to do so. This is primarily due to the large amount of coupling between Drone pipelines, our infra, and our tooling.

Here is a list of the changes the PR includes:

  • Removed .drone.yml, and Drone references throughout the code base. This includes Makefile targets that were used exclusively for the Drone release process.
  • Added a pipeline that will build all plugins artifacts.
    • This will trigger on tags, pushes to master, pull requests to master, and workflow dispatch events.
    • Triggers not directly tied to a specific tag in git will generate a new version number (not tag) for the build. This version number includes the name of the nearest tag, the commit used, and a random number. In semver, these will always be considered prerelease tags.
    • This version is passed down into builds, except for Helm charts (which use whatever value is committed). I can add support for Helm charts, but it'll take some additional work that I don't see as super valuable right now.
  • Added a pipeline that will publish all built plugins artifacts for a given tag.
    • This will trigger on tags, as well as new tagged GitHub releases being cut. I'm not sure whether or not we want to trigger on this. If we don't then I can pull the associated code out very easily.
  • Makefile targets for releases will no longer run the clean target. This doesn't add much (or any) value in a completely ephemeral environment, and really screws up parallel build processes.
  • The VERSION variable in every Makefile can now be set via CLI. I also added some linkage to ensure that it is passed down to to Makefile calls inside of Dockerfile builds.
  • The Terraform provider publishing tool has been largely rewritten. The core logic is still the same, however it no longer directly interacts with AWS. It now operates on a local copy of our Terraform repos and artifacts, and leaves syncing this local copy to other steps in the publishing process.
  • The CI/CD process now makes use of four environments that match what we are working towards in teleport.e - build-stage, build-prod, publish-stage, and publish-prod. With this, production publishing access can be much more tightly controlled. Additionally, changes to the release process can be fully tested (less Houston integration) prior to merge. This should result in a much more reliable release process.

Here is the remaining work that needs to be put on the backlog:

  • Drone infra for this repo needs to be cleaned up. Not only does this include deactivating the repo in Drone, but also pulling out the sizable amount of AWS infra dedicated to Drone/Teleport plugins repo.
  • After talking with @r0mant, plugins no longer have the expectation of being able to cut a separate release for each one. The current Makefiles are built on this assumption, and because of this it is very difficult (and tedious) to make changes. We need to refactor this process at some point. I looked into this work and it doesn't seem too difficult, but I didn't want to scope creep and include the changes in this PR.
  • This repo still uses Houston. I think it would be pretty easy to migrate plugins to the release server, but again I didn't want to increase the scope of this work.

I still have a couple of corner cases to check, but I don't expect the PR to change much so I'm opening it for review now.

Copy link

@orca-security-us orca-security-us bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orca Security Scan Summary

Status Check Issues by priority
Passed Passed Infrastructure as Code high 0   medium 1   low 2   info 0 View in Orca
Failed Failed Vulnerabilities high 0   medium 1   low 0   info 0 View in Orca
Passed Passed Secrets high 0   medium 0   low 0   info 0 View in Orca

@fheinecke fheinecke force-pushed the fred/gha-migration-1 branch 3 times, most recently from 20bc867 to 53f27a2 Compare January 18, 2024 07:44
@camscale
Copy link
Contributor

camscale commented Feb 7, 2024

BTW, I use actionlint (https://github.com/rhysd/actionlint) to lint my workflows. It has some false positives on the runners we use so I'm not recommending we enforce it. One nice thing it does is run shell check on the shell fragments in the workflows.

You can easily install actionlint with go install github.com/rhysd/actionlint/cmd/actionlint@latest

@fheinecke fheinecke force-pushed the fred/gha-migration-1 branch from e916978 to 95afbf1 Compare February 15, 2024 23:20
@fheinecke fheinecke merged commit cb4d95e into master Feb 17, 2024
19 of 21 checks passed
@fheinecke fheinecke deleted the fred/gha-migration-1 branch February 17, 2024 04:50
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants