Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update source for NVML header downloads to NVIDIA owned repo #124

Merged
merged 1 commit into from
May 27, 2024

Conversation

klueska
Copy link
Contributor

@klueska klueska commented May 24, 2024

An example of running this now looks like:

$ make update-nvml-h
Found 9 NVML headers:

No.  Version   Upload Time
  1  12.5.39   2024-05-21T16:33:04
  2  12.4.127  2024-04-03T20:31:07
  3  12.4.99   2024-03-05T17:03:22
  4  12.3.101  2023-11-16T22:01:02
  5  12.3.52   2023-10-19T15:56:27
  6  12.2.140  2023-08-29T15:40:51
  7  12.2.128  2023-07-31T15:43:39
  8  12.2.81   2023-06-28T15:36:05
  9  12.1.105  2023-04-19T15:31:00

Pick an NVML header to update ([1]-9): 1

NVML version: 12.5.39

Updating nvml.h to 12.5.39 from https://gitlab.com/nvidia/headers/cuda-individual/nvml_dev/-/raw/v12.5.39/nvml.h ...
--2024-05-24 20:46:21--  https://gitlab.com/nvidia/headers/cuda-individual/nvml_dev/-/raw/v12.5.39/nvml.h
Resolving gitlab.com (gitlab.com)... 172.65.251.78, 2606:4700:90:0:f22e:fbec:5bed:a9b9
Connecting to gitlab.com (gitlab.com)|172.65.251.78|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 595956 (582K) [text/plain]
Saving to: ‘/home/kklues/go-nvml/gen/nvml/nvml.h’

/home/kklues/go-nvml/gen/nvml/nvml.h            100%[=====================================================================================================>] 581.99K  --.-KB/s    in 0.006s

2024-05-24 20:46:21 (100 MB/s) - ‘/home/kklues/go-nvml/gen/nvml/nvml.h’ saved [595956/595956]

Successfully updated nvml.h to 12.5.39.

@klueska klueska self-assigned this May 24, 2024
@klueska klueska requested a review from elezar May 24, 2024 20:45
@klueska klueska force-pushed the update-nvml-header-download branch from c0516c9 to 99b70d3 Compare May 24, 2024 20:48
Makefile Outdated
wget -qO - https://api.anaconda.org/package/nvidia/cuda-nvml-dev/files | \
$(JQ) '.[] | select(.attrs.subdir=="linux-64") | .version + "@" + .upload_time[:19] + "@" + .full_name' | \
update-nvml-h: NVML_DEV_HEADERS_INFO := $(shell \
wget -qO - https://gitlab.com/api/v4/projects/nvidia%2Fheaders%2Fcuda-individual%2Fnvml_dev/repository/tags | \
Copy link
Contributor

@XuehaiPan XuehaiPan May 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about using git clone rather than querying the RESTful API? Since we are querying a git repo.

$ git clone https://gitlab.com/nvidia/headers/cuda-individual/nvml_dev.git nvml-dev
$ cd nvml-dev
$ git tag --list 'v*' --sort='-version:refname' --format='%(refname:short)%09%(*authordate:format:%Y-%m-%d-%H:%M:%S)'
v12.5.39    2024-05-15-22:26:21
v12.4.127   2024-03-28-23:49:40
v12.4.99    2024-02-29-05:54:15
v12.3.101   2023-11-09-11:05:04
v12.3.52    2023-10-16-04:09:26
v12.2.140   2023-08-23-07:12:43
v12.2.128   2023-07-27-08:52:32
v12.2.81    2023-06-26-10:09:29
v12.1.105   2023-04-17-17:34:53
v12.1.55    2023-02-23-09:20:23
v12.0.140   2023-01-28-19:30:42
v12.0.76    2022-12-03-11:07:09
v11.8.86    2022-09-29-20:24:19
v11.7.91    2022-07-29-12:29:29
v11.7.50    2022-05-04-18:22:18
v11.6.55    2022-01-11-16:52:51
v11.5.50    2022-01-10-17:04:04
v11.4.120   2022-01-28-00:02:05

$ git checkout v12.5.39
Note: switching to 'v12.5.39'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 878f50f CUDA 12.5.0 nvML_DEV 12.5.39
$ cp -f nvml.h ../gen/nvml/nvml.h

@klueska
Copy link
Contributor Author

klueska commented May 26, 2024

I don't want to clone the whole repo just to pull one file out of it and then delete the repo. What advantage would cloning serve?

@XuehaiPan
Copy link
Contributor

XuehaiPan commented May 26, 2024

I don't want to clone the whole repo just to pull one file out of it and then delete the repo.

The repo only contains three files: nvml.h, LICENSE, README.md.

What advantage would cloning serve?

  1. It would be easier to compare differences between tags (if we set it as a git submodule).
  2. A git clone (Update source for NVML header downloads to NVIDIA owned repo #124 (comment) shows 18 tags) will pull more tags than RESTful API (Update source for NVML header downloads to NVIDIA owned repo #124 (comment) shows 9 tags).
  3. Drop dependency for wget and jq.

@klueska klueska force-pushed the update-nvml-header-download branch from 99b70d3 to 8c2b25e Compare May 27, 2024 09:40
@klueska
Copy link
Contributor Author

klueska commented May 27, 2024

I've updated my version to at least include all of the tags that it should (i.e. 18). @elezar what are your thoughts on using the REST API vs. cloning the whole repo?

@klueska
Copy link
Contributor Author

klueska commented May 27, 2024

I don't want to clone the whole repo just to pull one file out of it and then delete the repo.

The repo only contains three files: nvml.h, LICENSE, README.md.

It's not about the size of the repo, but about the cleanup that needs to get done if something goes wrong. I do not want to be including this as a submodule as we don't actually depend on it for anything other than getting the latest NVML header (when appropriate).

Copy link
Member

@elezar elezar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's use thie REST API for this change and consider switching as a follow-up.

I suppose toding a shallow-clone and then fetching the tags is an option. We can also clone to /tmp so as to not have long-lived changes.

@klueska klueska merged commit 0e815c7 into NVIDIA:main May 27, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants