Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GitLab support #18

Open
hnez opened this issue Sep 17, 2024 · 0 comments
Open

Add GitLab support #18

hnez opened this issue Sep 17, 2024 · 0 comments

Comments

@hnez
Copy link
Collaborator

hnez commented Sep 17, 2024

At the surface the GitLab runner and the GitHub runner look quite different, but there may be a way to add GitLab support to Forrest without too many hacks and without sacrificing features like Docker support and without having to run the runner client outside of the VMs (if these two details were not relevant to us we could build something using the custom executor).

The GitLab runner polls for jobs to run by sending POST requests to the (undocumented) /api/v4/jobs/request GitLab API endpoint:

$ gitlab-runner run-single --url "http://localhost:8080" --token "glrt-GNma-AAAAAAAAAAAAAAA" --executor "shell"
$ nc -l 8080
POST /api/v4/jobs/request HTTP/1.1
Host: localhost:8080
User-Agent: gitlab-runner development version (HEAD; go1.22.6; linux/arm)
Content-Length: 847
Accept: application/json
Content-Type: application/json
Private-Token: glrt-GNma-AAAAAAAAAAAAAAA
Accept-Encoding: gzip

{"info":{"name":"gitlab-runner","version":"development version","revision":"HEAD","platform":"linux","architecture":"arm","executor":"shell","shell":"bash","features":{"variables":true,"image":false,"services":false,"artifacts":true,"cache":true,"fallback_cache_keys":true,"shared":true,"upload_multiple_artifacts":true,"upload_raw_artifacts":true,"session":true,"terminal":true,"refspecs":true,"masking":true,"proxy":false,"raw_variables":true,"artifacts_exclude":true,"multi_build_steps":true,"trace_reset":true,"trace_checksum":true,"trace_size":true,"vault_secrets":true,"cancelable":true,"return_exit_code":true,"service_variables":false,"service_multiple_aliases":false,"image_executor_opts":false,"service_executor_opts":false,"cancel_gracefully":true},"config":{"gpus":""}},"token":"glrt-GNma-AAAAAAAAAAAAAAA","system_id":"s_f7b6304c4e01"}

When no job is pending the API will respond with 204 No Content, but if a job is pending it will assign the job to the runner and will send back a json-formatted job definition. The definition also contains a job-token that will be used for all further communication.

This means we can:

  1. Pretend to be a GitLab runner and poll the API endpoint mentioned above (when we have the capacity to run another job of the configured type).
  2. Spawn a VM if a job is available.
  3. Provide a HTTP server that will respond to POST requests to /api/v4/jobs/request with the job definition in 1). It will also have to proxy some other API requests to the actual GitLab instance, since job status updates are sent that way (git clone etc. are however not. The correct URLs are included in the job definition).
    This proxy can run either inside the VM or as part of the Forrest process.

I've stumbled upon this approach thanks to the collabora/gitlab-runner-rs project, which allows building custom GitLab runners like the collabora/lava-gitlab-runner. Most of the features there are however out of scope for us, since we actually want to use the official GitLab runner inside of the VMs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant