Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add GPU/Accelerator support to VMs in GCPMachineTemplate #1341

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add GPU/Accelerator support to VMs in GCPMachineTemplate #1341

wants to merge 1 commit into from

Conversation

jwmay2012
Copy link
Contributor

@jwmay2012 jwmay2012 commented Oct 22, 2024 •
edited
Loading

What type of PR is this?

/kind feature

What this PR does / why we need it:
Adds the ability to configure Guest Accelerators like GPUs in a GCPMachineTemplate
Fixes #289

Special notes for your reviewer:
Tested and creates machines with GPUs correctly. After installing drivers and nvidia container runtime on the node, was able to get the GPU to run successfully in a Pod.
If you try to use an accelerator on the wrong instance type it will have an instance reconcile error from GCP that describes the improper API use.

OnHostMaintenance must be set to TERMINATE for GPU enabled machines.
https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_instance#guest_accelerator
Confirmed this is correct. Instance reconcile is rejected by GCP otherwise.
I set this field automatically.

TODOs:

  • squashed commits
  • includes documentation
  • adds unit tests

Release note:

Add GPU/Accelerator support for VMs in GCPMachineTemplate

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/feature Categorizes issue or PR as related to a new feature. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Oct 22, 2024
@k8s-ci-robot k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Oct 22, 2024
@k8s-ci-robot
Copy link
Contributor

Hi @jwmay2012. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Oct 22, 2024
@netlify Netlify
Copy link

netlify bot commented Oct 22, 2024 •
edited
Loading

✅ Deploy Preview for kubernetes-sigs-cluster-api-gcp ready!

Name Link
🔨 Latest commit 2358039
🔍 Latest deploy log https://app.netlify.com/sites/kubernetes-sigs-cluster-api-gcp/deploys/67644b37c92ef600083a6d44
😎 Deploy Preview https://deploy-preview-1341--kubernetes-sigs-cluster-api-gcp.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@salasberryfin
Copy link
Contributor

Thanks @jwmay2012

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Oct 22, 2024
@richardcase
Copy link
Member

@jwmay2012 - would you be able to run make lint on this change?

@reyvonger
Copy link
Contributor

Could you please provide an estimate of when this change might be included in a release?

salasberryfin
salasberryfin reviewed Dec 19, 2024
View reviewed changes
Copy link
Contributor

@salasberryfin salasberryfin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a minor comment, otherwise it looks good to me.

cloud/scope/machine.go
@@ -457,6 +473,11 @@ func (m *MachineScope) InstanceSpec(log logr.Logger) *compute.Instance {
instance.Metadata = m.InstanceAdditionalMetadataSpec()
instance.ServiceAccounts = append(instance.ServiceAccounts, m.InstanceServiceAccountsSpec())
instance.NetworkInterfaces = append(instance.NetworkInterfaces, m.InstanceNetworkInterfaceSpec())
instance.GuestAccelerators = m.InstanceGuestAcceleratorsSpec()
if len(instance.GuestAccelerators) > 0 {
instance.Scheduling.OnHostMaintenance = "TERMINATE"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a note to the docs on the value of OnHostMaintenance? I'd suggest we let users know that it will always default to TERMINATE.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏻 Added

@salasberryfin
Copy link
Contributor

Thanks @jwmay2012.

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 2, 2025
@jwmay2012
Copy link
Contributor Author

We good to merge? Been running a custom CAPG with these changes for a while and would love to get this upstream :)

@damdo
Copy link
Member

damdo commented Jan 27, 2025

@richardcase are you happy with this? If so would you be able to stamp your approval on it?
Thanks!

cpanato
cpanato approved these changes Jan 27, 2025
View reviewed changes
Copy link
Member

@cpanato cpanato left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/hold defer to @richardcase

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 27, 2025
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cpanato, jwmay2012

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@salasberryfin salasberryfin salasberryfin left review comments

@cpanato cpanato cpanato approved these changes

@dims dims Awaiting requested review from dims

@richardcase richardcase Awaiting requested review from richardcase

Assignees

@cpanato cpanato

@salasberryfin salasberryfin

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

GPU support
7 participants
@jwmay2012 @k8s-ci-robot @salasberryfin @richardcase @reyvonger @damdo @cpanato