Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add MOM6 support (om4 025jra ryf) #258

Merged
merged 46 commits into from
Nov 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
1d040c7
Factor out the 'static' frequency tag as a variable
marc-white Aug 20, 2024
cb8b017
Initial creation of AccessOm4Builder and test data structure
marc-white Aug 20, 2024
7e14a52
Correct exclude pattern, add Om4Builder to test data suite
marc-white Aug 20, 2024
2428c2d
Improved OM4 test data (still doesn't work though)
marc-white Aug 21, 2024
a13de8b
Fixed test data, fixed filename regexp
marc-white Aug 22, 2024
ed2ba42
Add FIXME for timestamp part of PATTERN regex
marc-white Aug 22, 2024
463005f
Add test_builder_parser tests for OM4
marc-white Aug 23, 2024
6829eb0
Add test_parse_access_filename tests for OM4
marc-white Aug 23, 2024
6796f42
Final tests for AccessOM4 builder (plus another TODO)
marc-white Aug 26, 2024
138e2a6
Add new pattern for OM4; add ability to have multiple redacted 'times…
marc-white Aug 27, 2024
79763cc
OM4 test expansion
marc-white Aug 28, 2024
3212a99
Added test data for panan-01-zstar data
marc-white Aug 28, 2024
b6e9d41
Refactor AccessOm4Builder --> Mom6Builder
marc-white Aug 28, 2024
2956b7b
Merge remote-tracking branch 'origin/main' into 175-data-request-add-…
marc-white Sep 24, 2024
0d476c6
Add first MOM6 datasets
marc-white Sep 24, 2024
24d97eb
Merge remote-tracking branch 'origin/main' into 175-data-request-add-…
marc-white Oct 1, 2024
b3ab3bd
Merge branch 'main' into 175-data-request-add-om4_025jra_ryf
marc-white Oct 1, 2024
120d92c
Merge branch 'main' into 175-data-request-add-om4_025jra_ryf
marc-white Oct 13, 2024
720cad1
Add test print for debugging
marc-white Oct 15, 2024
3fa73e7
Remove asset print debug
marc-white Oct 15, 2024
fc582ce
Add better error statement to CatalogManager __init__
marc-white Oct 15, 2024
6e7a0e9
Expand out ParserError message in validate_parser
marc-white Oct 15, 2024
fa89393
Tweak ParserError message
marc-white Oct 15, 2024
d428726
Further expand ParserError message
marc-white Oct 15, 2024
bf9450d
Add first pass at MOM6 translator (hack job)
marc-white Oct 17, 2024
6f31a53
Next pass at MOM6 translator
marc-white Oct 17, 2024
a95194c
Remembered to change the configured Translator
marc-white Oct 17, 2024
0eb2a97
Try MOM6 translator again...
marc-white Oct 17, 2024
581e9ff
Merge remote-tracking branch 'origin/main' into 175-data-request-add-…
marc-white Oct 20, 2024
b7856aa
Fix up MOM6 builder after last merge
marc-white Oct 21, 2024
2033537
Update test_builders for MOM6 - new parse_access_ncfile return
marc-white Oct 21, 2024
771978d
Try again without a hard-coded MOM6 model name
marc-white Oct 21, 2024
9e54ff3
Return model translator for MOM6
marc-white Oct 21, 2024
b6158ad
Merge branch 'main' into 175-data-request-add-om4_025jra_ryf
marc-white Nov 13, 2024
3531397
Fix MOM6 test variables etc.
marc-white Nov 14, 2024
4aac81b
Added docstring to parse_access_ncfile to make issue clear & updated …
charles-turner-1 Nov 18, 2024
6dfbd89
Merge branch 'main' into 175-data-request-add-om4_025jra_ryf
marc-white Nov 18, 2024
47166db
Remove redundant MOM6 translator
marc-white Nov 19, 2024
4331abc
PR updates to builders.py
marc-white Nov 19, 2024
5705d33
Update to except statement in CatalogManager
marc-white Nov 19, 2024
97f18db
Merge remote-tracking branch 'origin/main' into 175-data-request-add-…
marc-white Nov 25, 2024
bdb309b
Merge branch 'main' into 175-data-request-add-om4_025jra_ryf
rbeucher Nov 26, 2024
101c948
Ruff fix
marc-white Nov 26, 2024
cc2f6e5
Improve test coverage of Builder parser exceptions
marc-white Nov 26, 2024
2637426
Improve manager test coverage
marc-white Nov 26, 2024
428f0d2
Improve builders test coverage
marc-white Nov 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions config/mom6.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
builder: Mom6Builder

translator: DefaultTranslator

sources:

- metadata_yaml: /g/data/ik11/outputs/mom6-om4-025/OM4_025.JRA_RYF/metadata.yaml
path:
- /g/data/ik11/outputs/mom6-om4-025/OM4_025.JRA_RYF

- metadata_yaml: /g/data/ik11/outputs/mom6-panan/panant-01-hycom1-v13/metadata.yaml
path:
- /g/data/ik11/outputs/mom6-panan/panant-01-hycom1-v13

- metadata_yaml: /g/data/ik11/outputs/mom6-panan/panant-01-zstar-v13/metadata.yaml
path:
- /g/data/ik11/outputs/mom6-panan/panant-01-zstar-v13
18 changes: 11 additions & 7 deletions src/access_nri_intake/catalog/manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@

import intake
from intake_dataframe_catalog.core import DfFileCatalog, DfFileCatalogError
from pandas.errors import EmptyDataError

from ..utils import validate_against_schema
from . import (
Expand Down Expand Up @@ -46,13 +47,16 @@ def __init__(self, path: str):

self.mode = "a" if os.path.exists(path) else "w"

self.dfcat = DfFileCatalog(
path=self.path,
yaml_column=YAML_COLUMN,
name_column=NAME_COLUMN,
mode=self.mode,
columns_with_iterables=COLUMNS_WITH_ITERABLES,
)
try:
self.dfcat = DfFileCatalog(
path=self.path,
yaml_column=YAML_COLUMN,
name_column=NAME_COLUMN,
mode=self.mode,
columns_with_iterables=COLUMNS_WITH_ITERABLES,
)
except (EmptyDataError, DfFileCatalogError) as e:
raise Exception(str(e) + f": {self.path}") from e

self.source = None
self.source_metadata = None
Expand Down
89 changes: 84 additions & 5 deletions src/access_nri_intake/source/builders.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,15 +28,19 @@
"_mon$": (1, "mon"),
"1mon": (1, "mon"),
"yearly": (1, "yr"),
"annual": (1, "yr"),
"_ann$": (1, "yr"),
}

# ACCESS output file patterns
PATTERNS_HELPERS = {
"not_multi_digit": "(?:\\d(?!\\d)|[^\\d](?=\\d)|[^\\d](?!\\d))",
"om3_components": "(?:cice|mom6|ww3)",
"mom6_components": "(?:ocean|ice)",
"mom6_added_timestamp": "(\\d{4}_\\d{3})",
"ymds": "\\d{4}[_,\\-]\\d{2}[_,\\-]\\d{2}[_,\\-]\\d{5}",
"ymd": "\\d{4}[_,\\-]\\d{2}[_,\\-]\\d{2}",
"ymd-ns": "\\d{4}\\d{2}\\d{2}",
"ym": "\\d{4}[_,\\-]\\d{2}",
"y": "\\d{4}",
}
Expand Down Expand Up @@ -172,7 +176,10 @@ def validate_parser(self):
return self

raise ParserError(
"Parser returns no valid assets. Try parsing a single file with Builder.parser(file)"
f"""Parser returns no valid assets.
Try parsing a single file with Builder.parser(file)
Last failed asset: {asset}
Asset parser return: {info}"""
)

def build(self):
Expand Down Expand Up @@ -262,11 +269,13 @@ def parse_access_filename(
for pattern in patterns:
match = re.match(pattern, file_id)
if match:
# FIXME switch to using named group for timestamp
# Loop over all found groups and redact
timestamp = match.group(1)
redaction = re.sub(r"\d", redaction_fill, timestamp)
file_id = (
file_id[: match.start(1)] + redaction + file_id[match.end(1) :]
)
for grp in match.groups():
if grp is not None:
redaction = re.sub(r"\d", redaction_fill, grp)
file_id = re.sub(grp, redaction, file_id)
break

# Remove non-python characters from file ids
Expand Down Expand Up @@ -467,6 +476,76 @@ def parser(cls, file) -> dict:
return {INVALID_ASSET: file, TRACEBACK: traceback.format_exc()}


# FIXME refactor to be called Mom6Builder (TBC)
class Mom6Builder(BaseBuilder):
"""Intake-ESM datastore builder for MOM6 COSIMA datasets"""

# FIXME should be able to make one super-pattern, but couldn't
# make it work with the ? selector after mom6_added_timestamp
# NOTE: Order here is important!
PATTERNS = [
rf"[^\.]*({PATTERNS_HELPERS['ymd-ns']})\.{PATTERNS_HELPERS['mom6_components']}.*{PATTERNS_HELPERS['mom6_added_timestamp']}.*$", # Daily snapshot naming
rf"[^\.]*({PATTERNS_HELPERS['ymd-ns']})\.{PATTERNS_HELPERS['mom6_components']}.*$", # Basic naming
]

def __init__(self, path):
"""
Initialise a Mom6Builder

Parameters
----------
path : str or list of str
Path or list of paths to crawl for assets/files.
"""

kwargs = dict(
path=path,
depth=1,
exclude_patterns=[
"*restart*",
"*MOM_IC.nc",
"*sea_ice_geometry.nc",
"*ocean_geometry.nc",
"*ocean.stats.nc",
"*Vertical_coordinate.nc",
],
include_patterns=["*.nc"],
data_format="netcdf",
groupby_attrs=["file_id", "frequency"],
aggregations=[
{
"type": "join_existing",
"attribute_name": "start_date",
"options": {
"dim": "time",
"combine": "by_coords",
},
},
],
)

super().__init__(**kwargs)

@classmethod
def parser(cls, file):
try:
output_nc_info = cls.parse_access_ncfile(file)
ncinfo_dict = output_nc_info.to_dict()

if "ocean" in ncinfo_dict["filename"]:
realm = "ocean"
elif "ice" in ncinfo_dict["filename"]:
realm = "seaIce"
else:
raise ParserError(f"Cannot determine realm for file {file}")
ncinfo_dict["realm"] = realm

return ncinfo_dict

except Exception:
return {INVALID_ASSET: file, TRACEBACK: traceback.format_exc()}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codecov is complaining that this line isn't tested - tbh I think it's unimportant.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm assuming the equivalent lines aren't tested in the other Builders, but I might look into that.



class AccessEsm15Builder(BaseBuilder):
"""Intake-ESM datastore builder for ACCESS-ESM1.5 datasets"""

Expand Down
8 changes: 5 additions & 3 deletions src/access_nri_intake/source/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@
import cftime
import xarray as xr

FREQUENCY_STATIC = "fx"


class EmptyFileError(Exception):
pass
Expand Down Expand Up @@ -176,7 +178,7 @@ def _todate(t):
time_format = "%Y-%m-%d, %H:%M:%S"
ts = None
te = None
frequency: str | tuple[int | None, str] = "fx"
frequency: str | tuple[int | None, str] = FREQUENCY_STATIC
has_time = time_dim in ds

if has_time:
Expand Down Expand Up @@ -224,11 +226,11 @@ def _todate(t):
f"The frequency '{filename_frequency}' determined from filename does not "
f"match the frequency '{frequency}' determined from the file contents."
)
if frequency == "fx":
if frequency == FREQUENCY_STATIC:
frequency = filename_frequency
warnings.warn(f"{msg} Using '{frequency}'.")

if has_time & (frequency != "fx"):
if has_time & (frequency != FREQUENCY_STATIC):
if not has_bounds:
ts, te = _guess_start_end_dates(ts, te, frequency)

Expand Down
29 changes: 29 additions & 0 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,12 +46,41 @@ def pytest_collection_modifyitems(config, items):
test_builders::test_parse_access_ncfile when we check the file contents & to
ensure we correctly get xfails if we don't have cordinate discovery enabled
in intake-esm.

The commented out lines are tests which *could* reasonably be expected to fail,
but don't because they don't have any unused coordinates.
"""
for item in items:
if (
item.name
in (
"test_parse_access_ncfile[AccessOm2Builder-access-om2/output000/ocean/ocean_grid.nc-expected0-True]",
# "test_parse_access_ncfile[AccessOm2Builder-access-om2/output000/ocean/ocean.nc-expected1-True]",
# "test_parse_access_ncfile[AccessOm2Builder-access-om2/output000/ocean/ocean_month.nc-expected2-True]",
# "test_parse_access_ncfile[AccessOm2Builder-access-om2/output000/ocean/ocean_month_inst_nobounds.nc-expected3-True]",
# "test_parse_access_ncfile[AccessOm2Builder-access-om2/output000/ice/OUTPUT/iceh.1900-01.nc-expected4-True]",
# "test_parse_access_ncfile[AccessCm2Builder-access-cm2/by578/history/atm/netCDF/by578a.pd201501_dai.nc-expected5-True]",
# "test_parse_access_ncfile[AccessCm2Builder-access-cm2/by578/history/ice/iceh_d.2015-01.nc-expected6-True]",
# "test_parse_access_ncfile[AccessCm2Builder-access-cm2/by578/history/ocn/ocean_daily.nc-20150630-expected7-True]",
# "test_parse_access_ncfile[AccessCm2Builder-access-cm2/by578/history/ocn/ocean_scalar.nc-20150630-expected8-True]",
# "test_parse_access_ncfile[AccessEsm15Builder-access-esm1-5/history/atm/netCDF/HI-C-05-r1.pa-185001_mon.nc-expected9-True]",
# "test_parse_access_ncfile[AccessEsm15Builder-access-esm1-5/history/ice/iceh.1850-01.nc-expected10-True]",
# "test_parse_access_ncfile[AccessEsm15Builder-access-esm1-5/history/ocn/ocean_bgc_ann.nc-18501231-expected11-True]",
# "test_parse_access_ncfile[AccessEsm15Builder-access-esm1-5/history/ocn/ocean_bgc.nc-18501231-expected12-True]",
# "test_parse_access_ncfile[AccessOm3Builder-access-om3/output000/GMOM_JRA_WD.mom6.h.native_1900_01.nc-expected13-True]",
# "test_parse_access_ncfile[AccessOm3Builder-access-om3/output000/GMOM_JRA_WD.mom6.h.sfc_1900_01_02.nc-expected14-True]",
# "test_parse_access_ncfile[AccessOm3Builder-access-om3/output000/GMOM_JRA_WD.mom6.h.static.nc-expected15-True]",
# "test_parse_access_ncfile[AccessOm3Builder-access-om3/output000/GMOM_JRA_WD.mom6.h.z_1900_01.nc-expected16-True]",
# "test_parse_access_ncfile[AccessOm3Builder-access-om3/output000/GMOM_JRA_WD.cice.h.1900-01-01.nc-expected17-True]",
# "test_parse_access_ncfile[AccessOm3Builder-access-om3/output000/GMOM_JRA_WD.ww3.hi.1900-01-02-00000.nc-expected18-True]",
"test_parse_access_ncfile[Mom6Builder-mom6/output000/19000101.ice_daily.nc-expected19-True]",
"test_parse_access_ncfile[Mom6Builder-mom6/output000/19000101.ocean_annual_z.nc-expected20-True]",
"test_parse_access_ncfile[Mom6Builder-mom6/output000/19000101.ocean_month_rho2.nc-expected21-True]",
# "test_parse_access_ncfile[Mom6Builder-mom6/output000/19000101.ocean_scalar_annual.nc-expected22-True]",
"test_parse_access_ncfile[Mom6Builder-mom6/output000/19000101.ocean_static.nc-expected23-True]",
# "test_parse_access_ncfile[Mom6Builder-mom6/output053/20051101.ocean_daily_2005_360.nc-expected24-True]",
"test_parse_access_ncfile[Mom6Builder-mom6/output053/20051101.ocean_daily_rho2_2005_360.nc-expected25-True]",
"test_parse_access_ncfile[Mom6Builder-mom6/output053/20051101.ocean_daily_z_2005_360.nc-expected26-True]",
)
and _add_xfail
):
Expand Down
39 changes: 39 additions & 0 deletions tests/data/mom6/README
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
Generated using the following Python on NCI ARE:

```
import netCDF4
import glob
import os

srcdir = "/g/data/ik11/outputs/mom6-om4-025/OM4_025.JRA_RYF/"
destdir = "/home/120/mcw120/scratch/"

targets = ["output000", "output001"]
target_glob = "190*.nc"

for target in targets:
srcglob = os.path.join(srcdir, target, target_glob)
srcfns = [os.path.basename(_) for _ in glob.glob(srcglob)]
for srcfn in srcfns:
with netCDF4.Dataset(os.path.join(srcdir, target, srcfn), "r") as old, netCDF4.Dataset(os.path.join(destdir, target, srcfn), "w") as new:
print(old.filepath())
print(new.filepath())
new.setncattr("file_format", old.file_format)
new.setncatts(old.__dict__)
# new.set_fill_off()
for name, dimension in old.dimensions.items():
if name == "nv":
dimn = 2
elif not dimension.isunlimited():
dimn = 1
else:
dimn = dimension.size
new.createDimension(name, dimn)
for name, variable in old.variables.items():
x = new.createVariable(name, variable.datatype, variable.dimensions)
new[name].setncatts(old[name].__dict__)
if name == "time" or name == "nv": # Need time-related data only
new[name][:] = old[name][:]
print(">> Complete!")
print("--")
```
93 changes: 93 additions & 0 deletions tests/data/mom6/error_logs/env.28342543.gadi-pbs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
CPATH: /apps/openmpi/4.0.3/include
CPATH_modshare: /apps/openmpi/4.0.3/include:1
CPLUS_INCLUDE_PATH: /apps/openmpi/4.0.3/include
CPLUS_INCLUDE_PATH_modshare: /apps/openmpi/4.0.3/include:1
C_INCLUDE_PATH: /apps/openmpi/4.0.3/include
C_INCLUDE_PATH_modshare: /apps/openmpi/4.0.3/include:1
ENVIRONMENT: BATCH
FPATH: /apps/openmpi/4.0.3/include
FPATH_modshare: /apps/openmpi/4.0.3/include:1
GIT_CONFIG_NOGLOBAL: 'yes'
HOME: /home/157/amh157
LC_CTYPE: C.UTF-8
LD_LIBRARY_PATH: /apps/openmpi/4.0.3/lib:/apps/openmpi/4.0.3/lib/profilers
LD_LIBRARY_PATH_modshare: /apps/openmpi/4.0.3/lib:1:/apps/openmpi/4.0.3/lib/profilers:1
LD_RUN_PATH: /apps/openmpi/4.0.3/lib:/apps/openmpi/4.0.3/lib/profilers
LD_RUN_PATH_modshare: /apps/openmpi/4.0.3/lib:1:/apps/openmpi/4.0.3/lib/profilers:1
LIBRARY_PATH: /apps/openmpi/4.0.3/lib:/apps/openmpi/4.0.3/lib/profilers
LIBRARY_PATH_modshare: /apps/openmpi/4.0.3/lib:1:/apps/openmpi/4.0.3/lib/profilers:1
LOADEDMODULES: pbs:openmpi/4.0.3
LOADEDMODULES_modshare: pbs:1:openmpi/4.0.3:1
LOGNAME: amh157
MANPATH: /apps/openmpi/4.0.3/share/man:/opt/pbs/default/share/man
MANPATH_modshare: /apps/openmpi/4.0.3/share/man:1:/opt/pbs/default/share/man:1
MODULEPATH: /g/data3/hh5/public/modules:/opt/Modules/modulefiles:/opt/Modules/v4.3.0/modulefiles:/apps/Modules/modulefiles
MODULESHOME: /opt/Modules/v4.3.0
MODULES_CMD: /opt/Modules/v4.3.0/libexec/modulecmd.tcl
MODULES_LMCONFLICT: openmpi/4.0.3&mpi&lam&mpich&openmpi&intel-mpi&o/wrappers&o/yes-wrappers&o/use-wrappers&o/enable-wrappers&o/with-wrappers&o/no-wrappers&o/not-wrappers&o/disable-wrappers&o/without-wrappers&o/ld_library_path&o/yes-ld_library_path&o/use-ld_library_path&o/enable-ld_library_path&o/with-ld_library_path&o/no-ld_library_path&o/not-ld_library_path&o/disable-ld_library_path&o/without-ld_library_path&o/ld_run_path&o/yes-ld_run_path&o/use-ld_run_path&o/enable-ld_run_path&o/with-ld_run_path&o/no-ld_run_path&o/not-ld_run_path&o/disable-ld_run_path&o/without-ld_run_path&o/show-debug&o/yes-show-debug&o/use-show-debug&o/enable-show-debug&o/with-show-debug&o/no-show-debug&o/not-show-debug&o/disable-show-debug&o/without-show-debug&o/append-paths&o/yes-append-paths&o/use-append-paths&o/enable-append-paths&o/with-append-paths&o/no-append-paths&o/not-append-paths&o/disable-append-paths&o/without-append-paths&o/library_path&o/yes-library_path&o/use-library_path&o/enable-library_path&o/with-library_path&o/no-library_path&o/not-library_path&o/disable-library_path&o/without-library_path&o/packaged-envvars&o/yes-packaged-envvars&o/use-packaged-envvars&o/enable-packaged-envvars&o/with-packaged-envvars&o/no-packaged-envvars&o/not-packaged-envvars&o/disable-packaged-envvars&o/without-packaged-envvars
MODULES_LMCONFLICT_modshare: openmpi/4.0.3&mpi&lam&mpich&openmpi&intel-mpi&o/wrappers&o/yes-wrappers&o/use-wrappers&o/enable-wrappers&o/with-wrappers&o/no-wrappers&o/not-wrappers&o/disable-wrappers&o/without-wrappers&o/ld_library_path&o/yes-ld_library_path&o/use-ld_library_path&o/enable-ld_library_path&o/with-ld_library_path&o/no-ld_library_path&o/not-ld_library_path&o/disable-ld_library_path&o/without-ld_library_path&o/ld_run_path&o/yes-ld_run_path&o/use-ld_run_path&o/enable-ld_run_path&o/with-ld_run_path&o/no-ld_run_path&o/not-ld_run_path&o/disable-ld_run_path&o/without-ld_run_path&o/show-debug&o/yes-show-debug&o/use-show-debug&o/enable-show-debug&o/with-show-debug&o/no-show-debug&o/not-show-debug&o/disable-show-debug&o/without-show-debug&o/append-paths&o/yes-append-paths&o/use-append-paths&o/enable-append-paths&o/with-append-paths&o/no-append-paths&o/not-append-paths&o/disable-append-paths&o/without-append-paths&o/library_path&o/yes-library_path&o/use-library_path&o/enable-library_path&o/with-library_path&o/no-library_path&o/not-library_path&o/disable-library_path&o/without-library_path&o/packaged-envvars&o/yes-packaged-envvars&o/use-packaged-envvars&o/enable-packaged-envvars&o/with-packaged-envvars&o/no-packaged-envvars&o/not-packaged-envvars&o/disable-packaged-envvars&o/without-packaged-envvars:1
MODULE_VERSION: v4.3.0
MODULE_VERSION_STACK: v4.3.0
MXM_LOG_FILE: /dev/null
MXM_LOG_LEVEL: ERROR
NCPUS: '48'
OMPI_BASE: /apps/openmpi/4.0.3
OMPI_MCA_orte_tmpdir_base: /jobfs/28342543.gadi-pbs
OMPI_ROOT: /apps/openmpi/4.0.3
OMPI_VERSION: 4.0.3
OMP_NUM_THREADS: '48'
OPENMPI_BASE: /apps/openmpi/4.0.3
OPENMPI_ROOT: /apps/openmpi/4.0.3
OPENMPI_VERSION: 4.0.3
PATH: /apps/openmpi/wrapper/fortran:/apps/openmpi/wrapper:/apps/openmpi/4.0.3/bin:/bin:/usr/bin:/opt/pbs/default/bin
PATH_modshare: /apps/openmpi/4.0.3/bin:1:/bin:1:/apps/openmpi/wrapper/fortran:1:/usr/bin:1:/apps/openmpi/wrapper:1:/opt/pbs/default/bin:1
PAYU_PATH: /g/data3/hh5/public/apps/miniconda3/envs/analysis3-21.07/bin
PBS_ENVIRONMENT: PBS_BATCH
PBS_JOBCOOKIE: 7366EFF670A4AAA3664067773F6676EF
PBS_JOBDIR: /home/157/amh157
PBS_JOBFS: /jobfs/28342543.gadi-pbs
PBS_JOBID: 28342543.gadi-pbs
PBS_JOBNAME: OM4_025.JRA
PBS_MOMPORT: '15003'
PBS_NCI_FS_GDATA1: '0'
PBS_NCI_FS_GDATA1A: '0'
PBS_NCI_FS_GDATA1B: '0'
PBS_NCI_FS_GDATA2: '0'
PBS_NCI_FS_GDATA3: '0'
PBS_NCI_FS_GDATA4: '0'
PBS_NCI_HT: '0'
PBS_NCI_IMAGE: ''
PBS_NCI_JOBFS: 10gb
PBS_NCI_LAUNCH_COMPATIBILITY: '0'
PBS_NCI_NCPUS_PER_NODE: '48'
PBS_NCI_NCPUS_PER_NUMA: '12'
PBS_NCI_NUMA_PER_NODE: '4'
PBS_NCI_STORAGE: scratch/x77+gdata/ua8+gdata/hh5+gdata/x77
PBS_NCI_WD: '1'
PBS_NCPUS: '1776'
PBS_NGPUS: '0'
PBS_NNODES: '37'
PBS_NODEFILE: /local/spool/pbs/aux/28342543.gadi-pbs
PBS_NODENUM: '0'
PBS_O_HOME: /home/157/amh157
PBS_O_HOST: gadi-login-02.gadi.nci.org.au
PBS_O_LANG: en_AU.UTF-8
PBS_O_LOGNAME: amh157
PBS_O_MAIL: /var/spool/mail/amh157
PBS_O_PATH: /apps/ncview/2.1.7/bin:/g/data3/hh5/public/apps/miniconda3/envs/analysis3-21.07/bin:/g/data3/hh5/public/apps/miniconda3/condabin:/home/157/amh157/.local/bin:/home/157/amh157/bin:/opt/pbs/default/bin:/opt/nci/bin:/opt/bin:/opt/Modules/v4.3.0/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/pbs/default/bin
PBS_O_QUEUE: normal
PBS_O_SHELL: /bin/bash
PBS_O_SYSTEM: Linux
PBS_O_TZ: :/etc/localtime
PBS_O_WORKDIR: /home/157/amh157/mom6/OM4_025.JRA
PBS_QUEUE: normal-exec
PBS_TASKNUM: '1'
PBS_VMEM: '7627861917696'
PROJECT: x77
SHELL: /opt/bin/nfsh
TMPDIR: /jobfs/28342543.gadi-pbs
USER: amh157
VT_MAX_FLUSHES: '0'
VT_PFORM_LDIR: /jobfs/28342543.gadi-pbs
_LMFILES_: /opt/Modules/modulefiles/pbs:/apps/Modules/modulefiles/openmpi/4.0.3
_LMFILES__modshare: /apps/Modules/modulefiles/openmpi/4.0.3:1:/opt/Modules/modulefiles/pbs:1
Loading
Loading