199 cordex xfails

[charles-turner-1]

• Add xfails on cordex translator (broken - see [DATA REQUEST] 20km regional projections for CORDEX-CMIP6 from the Queensland Future Climate Science Program #199).
• Update storage flags, correct package dir
• 'Robustify' a couple of tests

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.15%. Comparing base (3590e46) to head (9fa65d4).
Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #304   +/-   ##
=======================================
  Coverage   98.15%   98.15%           
=======================================
  Files          11       11           
  Lines        1028     1028           
=======================================
  Hits         1009     1009           
  Misses         19       19           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[marc-white]

Looks good to go, although it looks like e2e hasn't passed on this branch yet.

[charles-turner-1]

Looks good to go, although it looks like e2e hasn't passed on this branch yet.

Haven't updated the reference copy yet - good catch.

[charles-turner-1]

I'm still unable to update the reference repo at /g/data/xp65/admin/access-nri-intake-catalog & I've broken my git keys trying to fix it (see unverified badge above). I think that merging this & updating the reference repo should fix the issue.

NB. Last failed CI run actually passed: see

$ cd /g/data/xp65/admin/access-nri-intake-catalog 
$ cat test_end_to_end.sh.o130485707
...    array = array.get_duck_array()

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========== 108 passed, 1 xfailed, 57 warnings in 542.74s (0:09:02) ============

======================================================================================
                  Resource Usage on 2024-12-11 17:14:31:
   Job Id:             130485707.gadi-pbs
   Project:            iq82
   Exit Status:        0
   Service Units:      5.02
   NCPUs Requested:    12                     NCPUs Used: 12              
                                           CPU Time Used: 00:37:51        
   Memory Requested:   64.0GB                Memory Used: 64.0GB          
   Walltime requested: 00:30:00            Walltime Used: 00:09:25        
   JobFS requested:    100.0MB                JobFS used: 3.13MB          
======================================================================================

@rbeucher any ideas as to why this might be failing?

[charles-turner-1]

@marc-white @rbeucher

I think you may have somehow set some of the git refs to read-only for anyone but you - see

$ cd /g/data/xp65/admin/access-nri-intake-catalog/.git/logs/refs/remotes/origin 

[ct1163@gadi-login-01 origin]$ ll -t
total 180
-rw-rw-r--+ 1 ct1163 xp65_w  483 Dec 11 18:11 199-cordex-xfails
-rw-r--r--  1 mcw120 xp65_w  307 Dec 11 16:22 271-consolidation-on-pathlib-module *
-rw-r--r--  1 mcw120 xp65_w  308 Dec 11 16:22 pre-commit-ci-update-config *
-rw-rw-r--+ 1 ct1163 xp65_w  161 Dec 11 15:18 257-replace-mocks
-rw-rw-r--+ 1 ct1163 xp65_w  161 Dec 11 15:18 279-documentation-updates
-rw-rw-r--+ 1 ct1163 xp65_w  161 Dec 11 15:18 285-add-metadata-validation-tests
-rw-rw-r--+ 1 ct1163 xp65_w  161 Dec 11 15:18 286-bug-metadata-issues-in-v100b2
-rw-rw-r--+ 1 ct1163 xp65_w  161 Dec 11 15:18 290-align-test-datasets
-rw-rw-r--+ 1 ct1163 xp65_w  161 Dec 11 15:18 292-flaky-tests
-rw-rw-r--+ 1 ct1163 xp65_w  161 Dec 11 15:18 294-enable-catalog-version-reporting
-rw-rw-r--+ 1 ct1163 xp65_w  161 Dec 11 15:18 297-translator-errors
-rw-rw-r--+ 1 ct1163 xp65_w  161 Dec 11 15:18 299-doc-updates
-rw-rw-r--+ 1 ct1163 xp65_w  161 Dec 11 15:18 303-sanity-check-ipynb
-rw-rw-r--+ 1 rb5533 xp65_w  980 Dec 11 15:18 main
-rw-rw-r--+ 1 rb5533 xp65_w  322 Dec  3 11:16 153-e2e
-rw-rw-r--+ 1 rb5533 xp65_w  322 Dec  3 11:16 168-access-cm3-builder
-rw-rw-r--+ 1 rb5533 xp65_w  483 Dec  3 11:16 175-data-request-add-om4_025jra_ryf
-rw-r--r--  1 mcw120 xp65_w  154 Dec  3 11:16 265-bug-cli-tests-fail-if-not-top-level-directory *
-rw-r--r--  1 mcw120 xp65_w  154 Dec  3 11:16 270-teardown-of-temporary-file-structures *
-rw-r--r--  1 mcw120 xp65_w  154 Dec  3 11:16 273-remove-hh5-references *
-rw-r--r--  1 mcw120 xp65_w  154 Dec  3 11:16 277-v100-release *
-rw-r--r--  1 mcw120 xp65_w  154 Dec  3 11:16 283-bug-cordex-build-failing-in-v100b2 *
drwxr-sr-x  3 mcw120 xp65_w 4096 Dec  3 11:16 dependabot
-rw-r--r--  1 mcw120 xp65_w  154 Dec  3 11:16 rbeucher-patch-4 *
-rw-rw-r--+ 1 rb5533 xp65_w  168 Nov 21 12:40 184-data-request-add-cosima-experiments
-rw-rw-r--+ 1 rb5533 xp65_w  168 Nov 21 12:40 188-the-frequency-in-the-intake-catalog-does-not-account-for-snapshot-data
-rw-rw-r--+ 1 rb5533 xp65_w  168 Nov 21 12:40 191-default-to-a-latest-catalog-version
-rw-rw-r--+ 1 rb5533 xp65_w  168 Nov 21 12:40 216-make-metadata-validate-provide-a-full-report-on-first-pass
-rw-rw-r--+ 1 rb5533 xp65_w  168 Nov 21 12:40 223-model-required
-rw-rw-r--+ 1 rb5533 xp65_w  168 Nov 21 12:40 224-remove-missing-erai-metadatayaml
-rw-rw-r--+ 1 rb5533 xp65_w  168 Nov 21 12:40 235-config
-rw-rw-r--+ 1 rb5533 xp65_w  168 Nov 21 12:40 246-existing-catalog-that-doesnt-conform
-rw-rw-r--+ 1 rb5533 xp65_w  168 Nov 21 12:40 254-deprecate-py39
-rw-rw-r--+ 1 rb5533 xp65_w  168 Nov 21 12:40 263-bug-build-function-minmax
-rw-rw-r--+ 1 rb5533 xp65_w  168 Nov 21 12:40 issue176-om3-regex
-rw-rw-r--+ 1 rb5533 xp65_w  168 Nov 21 12:40 rbeucher-patch-1
-rw-rw-r--+ 1 rb5533 xp65_w  168 Nov 21 12:40 rbeucher-patch-2
-rw-rw-r--+ 1 rb5533 xp65_w  168 Nov 21 12:40 rbeucher-patch-3
-rw-rw-r--+ 1 rb5533 xp65_w  168 Nov 21 12:40 toggle-userdata
-rw-rw-r--+ 1 rb5533 xp65_w  168 Nov 21 12:40 164-update-metadata-schema-exp-1-0-3
-rw-rw-r--+ 1 rb5533 xp65_w  168 Oct  8 16:41 196-data-request-cmip6-era-regional-climate-model-data-ccam-acs
-rw-rw-r--+ 1 rb5533 xp65_w  330 Oct  8 16:41 660-coordinate-variables
-rw-rw-r--+ 1 rb5533 xp65_w  168 Oct  8 16:41 barpa_py18
-rw-rw-r--+ 1 rb5533 xp65_w  161 Oct  3 11:26 fix_metadata_yml
-rw-rw-r--+ 1 rb5533 xp65_w  206 Oct  1 12:01 HEAD

I've starred lines where permissions have gone awry... not quiet sure how this has happened but it seems to be whats blocking this.

[rbeucher]

It's @marc-white who is blocking, right? 😂 Yeah collaborative work on a local git repo is not a great idea...

[charles-turner-1]

Yup, tricky business, haha... Is it safe to delete & reclone the repo or is that going to make the situation worse rather than better?

[charles-turner-1]

There seems to be a single ref that's set to read only for you @rbeucher

-rw-r--r--  1 mcw120 xp65_w  154 Dec  3 11:16 rbeucher-patch-4 

Either way, I think we need to fix this before working out why the workflows are reporting as failed despite returning 0 - presumably some quirk of PBS I'm not aware of. One for tomorrow...

[rbeucher]

I'll have to fix that tomorrow. Laptop is in the office.

[marc-white]

I think you should be fine to just destroy the directory and re-clone it.

[charles-turner-1]

I'm satisfied that these tests run fine when called directly on Gadi - shall we merge this & open an issue for fixing the workflow?