Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker Multistage for parallel build #3048

Open
wants to merge 28 commits into
base: master
Choose a base branch
from

Conversation

th-2021
Copy link

@th-2021 th-2021 commented Jan 10, 2025

This PR moves the builds from install.sh into Dockerfile.multistage, allowing for parallel build execution (with docker buildkit).

This changes circleci configuration to use large modell (4 CPU, 15 GB memory). Also buildkit is enabled. This brings down build time to 48 min on circleci.

@sachilles and @kkimurak please review this.

The change is parallel to the existing files: Dockerfile.multistage and install2.sh are used.

@kkimurak
Copy link
Contributor

kkimurak commented Jan 10, 2025

Ok, it's done to build. It takes about 55m in total (47m to build).

First impression: Is it possible to separate multistage build and circleci config updates?
As far as I read the comments in the PR, I can see that just specifying "resource_class: large" and "use-buildkit: true" to circleci config will reduce the build time to less than an hour (I have never tested so I'll try later by myself..). If that's the case, we should merge this first. If we only need to make a few changes, the review time will be shorter and we will be able to start the release process quickly.

Regarding the multistage build, there are a lot of changes, and another comment #3042 (comment) pointed out that the image size has increased, so I think it would be a good idea to take some time to investigate this.

@th-2021
Copy link
Author

th-2021 commented Jan 10, 2025

Ok. Build worked within the same time range ~48 min, I'll separate circleci into a new PR.

@kkimurak
Copy link
Contributor

Great!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants