Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallel job execution #75

Merged
merged 5 commits into from
Jan 15, 2025
Merged

Parallel job execution #75

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
b03495f
Support starting mutliple instances of each runner manager
rahearn Jan 14, 2025
048eb1f
Update concurrent setting of gitlab-runner to 10 on startup
rahearn Jan 14, 2025
3f0dba0
Ensure we're only updating from concurrent = 1
rahearn Jan 14, 2025
3672023
Update compliance docs for multiple instances
rahearn Jan 14, 2025
f20137d
Make runner concurrency configurable via terraform env
rahearn Jan 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/compliance
Open in desktop
Submodule compliance updated from f0b938 to b65fa2
2 changes: 1 addition & 1 deletion main.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ resource "cloudfoundry_app" "gitlab-runner-manager" {
path = data.archive_file.src.output_path
source_code_hash = data.archive_file.src.output_base64sha256
buildpacks = ["https://github.com/cloudfoundry/apt-buildpack", "binary_buildpack"]
instances = 1
instances = var.manager_instances
command = "gitlab-runner run"
memory = var.manager_memory
health_check_type = "process"
Expand Down
1 change: 1 addition & 0 deletions runner-manager/.profile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@ if pgrep 'gitlab-runner' > /dev/null ; then
else
echo "Registering GitLab Runner with name $RUNNER_NAME"
if gitlab-runner register; then
sed -e 's/concurrent = 1$/concurrent = 10/' -i.bak .gitlab-runner/config.toml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[question] I'm wondering if this can be configured through the env like with the other settings? e.g., PREPARE_EXEC

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[question (nb)] how did you land on 10?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Um, yes, this script will have access to the env, I'll make this configurable now.

10 just seemed like a good balance between parallelism and not running out of memory quota

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure you can. I was trying to find the variable--the ones we use come from gitlab-runner register --help and concurrency doesn't seem to be listed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, I misunderstood.

No, this is the only way to set the concurrent setting, but gitlab-runner creates this file when you call register and reloads it when it changes. What I was going to do was update the concurrent = 10 to be concurrent = $OUR_ENV_SETTING so that we can still change this per-customer

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to you, why this doesn't have a setting?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really! Other than the counter argument that it does have a setting, just not via ENV vars. We could probably switch out all of our ENV var settings and just bundle a .toml file instead. If we have to add any more sed commands then we'll probably want to just go ahead and move everything to the file.

echo "GitLab Runner successfully registered"
else
exit_with_failure "GitLab Runner not registered"
Expand Down
1 change: 1 addition & 0 deletions sandbox-deploy/main.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ module "sandbox-runner" {
ci_server_token = var.ci_server_token
docker_hub_user = var.docker_hub_user
docker_hub_token = var.docker_hub_token
manager_instances = 1
developer_emails = var.developer_emails
worker_disk_size = var.worker_disk_size
worker_egress_allowlist = var.worker_egress_allowlist
Expand Down
6 changes: 6 additions & 0 deletions variables.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,12 @@ variable "runner_executor" {
description = "Runner Executer"
}

variable "manager_instances" {
type = number
default = 2
description = "Number of manager instances to run"
}

variable "manager_memory" {
type = number
default = 512
Expand Down
Loading