Add fre-nctools package #188

dougiesquire · 2025-02-04T05:46:08Z

This PR adds a package for FRE-NCtools. The initial implementation is taken and updated from COSIMA/spack_packages.

Closes #143

dougiesquire · 2025-02-04T23:12:44Z

I've tested building all versions on Gadi. Note, a number of releases are broken (see my comments in the package). I've only included versions that aren't broken. Alternatively, we could easily patch the broken versions, but that doesn't seem worth it to me... Lemme know

dougiesquire · 2025-02-04T23:16:19Z

Also, lol at the issue assignment chaos

packages/fre-nctools/package.py

aidanheerdegen · 2025-02-05T01:49:26Z

Also, lol at the issue assignment chaos

Don't let an old man near a computer.

packages/fre-nctools/package.py

anton-seaice · 2025-02-05T02:28:43Z

I'll defer to @harshula to approve unless im needed :-)

harshula · 2025-02-06T03:15:40Z

Hi @dougiesquire ,

Slight complication:

[root@de168857d600 packages]# spack install fre-nctools %[email protected] target=x86_64
...
==> Installing fre-nctools-2024.05-2v2mri5qpx4pgmvjgj4y5tdy5xysp7kg [52/52]
==> No binary for fre-nctools-2024.05-2v2mri5qpx4pgmvjgj4y5tdy5xysp7kg found: installing from source
==> Fetching https://github.com/NOAA-GFDL/FRE-NCtools/tarball/2024.05
==> Error: ChecksumError: sha256 checksum failed for /tmp/root/spack-stage/spack-stage-fre-nctools-2024.05-2v2mri5qpx4pgmvjgj4y5tdy5xysp7kg/2024.05
    Expected 61cec52aa03e066b64bed794ef9dc3eb28654c3d1b872aef1b69ce99ef7a9c65 but got 94f15eb1561951c537ffe93677e29b8975a9b0d8f7335e0dea3dbf95c9b5203b. File size = 190825505 bytes. Contents = b'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03\xec<kw\xdb6...~\xd6\xcf\xfa\xe1\xcf\xff\x07\x01\x8c\x06\xcc\x00\x88\x82,'

[root@de168857d600 packages]# spack install [email protected] %[email protected] target=x86_64
...
==> Installing fre-nctools-2024.05-2v2mri5qpx4pgmvjgj4y5tdy5xysp7kg [46/46]
==> No binary for fre-nctools-2024.05-2v2mri5qpx4pgmvjgj4y5tdy5xysp7kg found: installing from source
==> Fetching https://github.com/NOAA-GFDL/FRE-NCtools/tarball/2024.05
==> Error: ChecksumError: sha256 checksum failed for /tmp/root/spack-stage/spack-stage-fre-nctools-2024.05-2v2mri5qpx4pgmvjgj4y5tdy5xysp7kg/2024.05
    Expected 61cec52aa03e066b64bed794ef9dc3eb28654c3d1b872aef1b69ce99ef7a9c65 but got 94f15eb1561951c537ffe93677e29b8975a9b0d8f7335e0dea3dbf95c9b5203b. File size = 190825505 bytes. Contents = b'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03\xec<kw\xdb6...~\xd6\xcf\xfa\xe1\xcf\xff\x07\x01\x8c\x06\xcc\x00\x88\x82,'

[root@de168857d600 packages]# spack install [email protected] %[email protected] target=x86_64
...
==> Installing fre-nctools-2024.04-7k5eni4m24uney43fxhiqmszq4lwfkoh [39/39]
==> No binary for fre-nctools-2024.04-7k5eni4m24uney43fxhiqmszq4lwfkoh found: installing from source
==> Fetching https://github.com/NOAA-GFDL/FRE-NCtools/tarball/2024.04
==> Error: ChecksumError: sha256 checksum failed for /tmp/root/spack-stage/spack-stage-fre-nctools-2024.04-7k5eni4m24uney43fxhiqmszq4lwfkoh/2024.04
    Expected e27346d7ade1b67af163bb7f327a47a288d5e475fe797323bd7cee3a46385de0 but got c2db3aee5d4a30b18010997a4b953ed04e3b8981b39471652d39d6ae37042c47. File size = 321029281 bytes. Contents = b'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03\xec\\\xffw\xdb\xb8...\x0f\xbf\x87\xdf\xf2\xbf\xff\x1fX\xb4\xc3\x95\x00\x18\xd28'

$ wget https://github.com/NOAA-GFDL/FRE-NCtools/tarball/2024.04
--2025-02-06 13:29:04--  https://github.com/NOAA-GFDL/FRE-NCtools/tarball/2024.04
Resolving github.com (github.com)... 4.237.22.38
Connecting to github.com (github.com)|4.237.22.38|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/NOAA-GFDL/FRE-NCtools/legacy.tar.gz/refs/tags/2024.04 [following]
--2025-02-06 13:29:05--  https://codeload.github.com/NOAA-GFDL/FRE-NCtools/legacy.tar.gz/refs/tags/2024.04
Resolving codeload.github.com (codeload.github.com)... 4.237.22.35
Connecting to codeload.github.com (codeload.github.com)|4.237.22.35|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-gzip]
Saving to: ‘2024.04’

$ wget https://github.com/NOAA-GFDL/FRE-NCtools/archive/2024.04.tar.gz
--2025-02-06 13:36:11--  https://github.com/NOAA-GFDL/FRE-NCtools/archive/2024.04.tar.gz
Resolving github.com (github.com)... 4.237.22.38
Connecting to github.com (github.com)|4.237.22.38|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/NOAA-GFDL/FRE-NCtools/tar.gz/refs/tags/2024.04 [following]
--2025-02-06 13:36:12--  https://codeload.github.com/NOAA-GFDL/FRE-NCtools/tar.gz/refs/tags/2024.04
Resolving codeload.github.com (codeload.github.com)... 4.237.22.35
Connecting to codeload.github.com (codeload.github.com)|4.237.22.35|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-gzip]
Saving to: ‘2024.04.tar.gz’

$ ls -l 2024.04*
-rw-rw-r-- 1 [...] [...] 321029281 Feb  6 13:29 2024.04
-rw-rw-r-- 1 [...] [...] 321031930 Feb  6 13:36 2024.04.tar.gz

$ sha256sum 2024.04*
c2db3aee5d4a30b18010997a4b953ed04e3b8981b39471652d39d6ae37042c47  2024.04
e27346d7ade1b67af163bb7f327a47a288d5e475fe797323bd7cee3a46385de0  2024.04.tar.gz

It appears that the URL format we have adopted in our SPRs result in the downloaded tarball not matching the released tarball. Looking at the Spack Packaging document (https://spack.readthedocs.io/en/latest/packaging_guide.html#writing-a-package-recipe) they seem to use url_fmt = "https://github.com/uclouvain/openjpeg/archive/version.{0}.tar.gz" whereas we have used https://github.com/ISSMteam/ISSM/tarball/v{0}. For this SPR, please use the URL format that is used in the Spack Packaging Document. Hopefully, that will result in checksums matching with the released tarball checksums.

harshula · 2025-02-06T03:24:30Z

i.e. https://github.com/NOAA-GFDL/FRE-NCtools/archive/{0}.tar.gz

dougiesquire · 2025-02-06T04:06:32Z

@harshula, weird... Both formats seem to work for me? I've just updated the format. Does it work for you now?

harshula · 2025-02-06T04:28:22Z

Works for me now:

[root@de168857d600 packages]# git show
commit 6fa22d3d83f6ecb3fad5d6a61d984cf2320ab77d (HEAD -> 143-add-frenctools, origin/143-add-frenctools)
Author: dougiesquire <[email protected]>
Date:   Thu Feb 6 15:05:29 2025 +1100

    fre-nctools: change url format for tarballs

diff --git a/packages/fre-nctools/package.py b/packages/fre-nctools/package.py
index 415a65e..43d640f 100644
--- a/packages/fre-nctools/package.py
+++ b/packages/fre-nctools/package.py
@@ -47,7 +47,7 @@ class FreNctools(AutotoolsPackage):
     depends_on("nco", when="@2024.05:")
 
     def url_for_version(self, version):
-        return "https://github.com/NOAA-GFDL/FRE-NCtools/tarball/{0}".format(version)
+        return "https://github.com/NOAA-GFDL/FRE-NCtools/archive/{0}.tar.gz".format(version)
 
     def configure_args(self):
         spec = self.spec

[root@de168857d600 packages]# spack install [email protected] %[email protected] target=x86_64
...
==> Installing fre-nctools-2024.04-qjsaep5rk3zdxr2wt2ntxp7e4jpobtrh [39/39]
==> No binary for fre-nctools-2024.04-qjsaep5rk3zdxr2wt2ntxp7e4jpobtrh found: installing from source
==> Fetching https://github.com/NOAA-GFDL/FRE-NCtools/archive/2024.04.tar.gz
==> No patches needed for fre-nctools
==> fre-nctools: Executing phase: 'autoreconf'
==> fre-nctools: Executing phase: 'configure'
==> fre-nctools: Executing phase: 'build'
==> fre-nctools: Executing phase: 'install'
==> fre-nctools: Successfully installed fre-nctools-2024.04-qjsaep5rk3zdxr2wt2ntxp7e4jpobtrh
  Stage: 47.35s.  Autoreconf: 3.15s.  Configure: 5.43s.  Build: 37.15s.  Install: 0.58s.  Post-install: 0.12s.  Total: 1m 33.92s
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/fre-nctools-2024.04-qjsaep5rk3zdxr2wt2ntxp7e4jpobtrh

harshula

Thanks!

dougiesquire · 2025-02-06T04:31:20Z

@harshula would you like me to tidy up the commit history before merging?

dougiesquire · 2025-02-06T04:31:44Z

Oh wait, I see the default merge is "Squash and merge". Is that the preference?

harshula · 2025-02-06T04:53:06Z

Hi @dougiesquire , Yes, please "Squash and merge" and clean up the commit message.

Add FRE-NCtools package.

b8ebfbc

dougiesquire marked this pull request as draft February 4, 2025 11:23

fre-nctools: add new working versions

95571ef

aidanheerdegen assigned harshula Feb 4, 2025

harshula self-requested a review February 4, 2025 23:09

harshula assigned dougiesquire Feb 4, 2025

dougiesquire force-pushed the 143-add-frenctools branch from c7cd5fd to 95571ef Compare February 4, 2025 23:09

harshula removed their assignment Feb 4, 2025

dougiesquire marked this pull request as ready for review February 4, 2025 23:10

aidanheerdegen unassigned dougiesquire Feb 4, 2025

anton-seaice reviewed Feb 4, 2025

View reviewed changes

packages/fre-nctools/package.py Show resolved Hide resolved

packages/fre-nctools/package.py Show resolved Hide resolved

fre-nctools: add license

cc66b34

aidanheerdegen assigned dougiesquire Feb 5, 2025

fre-nctools: add url_for_version and "main" version

613cc3d

anton-seaice reviewed Feb 5, 2025

View reviewed changes

packages/fre-nctools/package.py Outdated Show resolved Hide resolved

packages/fre-nctools/package.py Show resolved Hide resolved

fre-nctools: fix typo in url_for_version

b96931d

fre-nctools: change url format for tarballs

6fa22d3

harshula approved these changes Feb 6, 2025

View reviewed changes

dougiesquire merged commit 0330062 into main Feb 6, 2025
1 check passed

dougiesquire deleted the 143-add-frenctools branch February 6, 2025 04:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add fre-nctools package #188

Add fre-nctools package #188

dougiesquire commented Feb 4, 2025

dougiesquire commented Feb 4, 2025 •

edited

Loading

dougiesquire commented Feb 4, 2025 •

edited

Loading

aidanheerdegen commented Feb 5, 2025

anton-seaice commented Feb 5, 2025

harshula commented Feb 6, 2025

harshula commented Feb 6, 2025

dougiesquire commented Feb 6, 2025

harshula commented Feb 6, 2025

harshula left a comment

dougiesquire commented Feb 6, 2025

dougiesquire commented Feb 6, 2025

harshula commented Feb 6, 2025

Add fre-nctools package #188

Add fre-nctools package #188

Conversation

dougiesquire commented Feb 4, 2025

dougiesquire commented Feb 4, 2025 • edited Loading

dougiesquire commented Feb 4, 2025 • edited Loading

aidanheerdegen commented Feb 5, 2025

anton-seaice commented Feb 5, 2025

harshula commented Feb 6, 2025

harshula commented Feb 6, 2025

dougiesquire commented Feb 6, 2025

harshula commented Feb 6, 2025

harshula left a comment

Choose a reason for hiding this comment

dougiesquire commented Feb 6, 2025

dougiesquire commented Feb 6, 2025

harshula commented Feb 6, 2025

dougiesquire commented Feb 4, 2025 •

edited

Loading

dougiesquire commented Feb 4, 2025 •

edited

Loading