Merge branch 'katec/derecho_update' into ktc/fixin_slap

builds/derecho-intel/README

@@ -0,0 +1,30 @@
+This directory contains posix-compliant CMake build scripts (*-cmake.sh) and
+shell agnostic build scripts (*-cmake), which are mostly intended for csh/tcsh
+users. 
+
+The posix scripts are recommended because they support out of source builds by
+using the first command-line argument to set the (top-level) CISM source
+directory. For example, to build in this directory, you may run the command:
+
+source PLATFORM-COMPILER-cmake.sh
+
+Likewise, if your CISM source is located in $CISM, you may build in a $BUILD
+subdirectory by:
+
+mkdir -p $BUILD && cd $BUILD source
+$CISM/builds/PLATFORM-COMPILER/PLATFORM-COMPILER-cmake.sh $CISM
+
+This functionality is necessary for using the build and test structure (BATS,
+located in $CISM/tests/regression/), will allow you to build multiple versions
+of CISM for testing, and make cleaning up the build directory easier.
+
+Because there isn't a reasonable way to make out-of-source capable build
+scripts work across both posix-compliant shells (sh, bash, zsh, etc.) and
+non-compliant shells (csh, tcsh), a shell agnostic build script is also
+available for csh/tcsh users. 
+
+NOTE: If you edit one of these script, please reflect your changes in the other
+script! 
+
+WARNING: It is possible these scripts will have diverged, so if you're having
+build problems, first check the differences between the two scripts. 

builds/derecho-intel/derecho-intel-cmake

@@ -0,0 +1,60 @@
+# Run this script by typing: source cheyenne-intel-cmake
+# After this script completes, type: make -j 8
+# If rebuilding, type 'make clean' before running 'make -j 8'
+
+# This cmake configuration script is set up to perform a parallel build with Trilinos
+
+module purge
+module load ncarenv/23.06
+module load intel/2023.0.0
+module load intel-mpi/2021.8.0
+module load mkl/2023.0.0
+#module load mpt/2.19
+module load netcdf/4.9.2
+module load ncarcompilers/1.0.0
+#module load pnetcdf/1.11.0
+module load cmake/3.26.3
+#module load python/2.7.13
+#module load numpy/1.12.0
+#module load netcdf4-python/1.2.7
+
+# remove old build data:
+rm -f ./CMakeCache.txt
+rm -rf ./CMakeFiles
+
+echo
+echo "Doing CMake Configuration step"
+
+# Note: the compilation flags were taken from the defaults for a CESM build on
+# cheyenne-intel (using cime at 84aafd5). Some of these options are probably
+# unnecessary for a standalone cism build, but I am keeping things consistent
+# with the CESM build for simplicity.
+
+cmake \
+  -D CISM_BUILD_CISM_DRIVER:BOOL=ON \
+  -D CISM_ENABLE_BISICLES=OFF \
+  -D CISM_ENABLE_FELIX=OFF \
+\
+  -D CISM_USE_TRILINOS:BOOL=OFF \
+  -D CISM_MPI_MODE:BOOL=ON \
+  -D CISM_SERIAL_MODE:BOOL=OFF \
+\
+  -D CISM_USE_GPTL_INSTRUMENTATION:BOOL=OFF \
+  -D CISM_COUPLED:BOOL=OFF \
+  -D CISM_USE_CISM_FRONT_END:BOOL=OFF \
+\
+  -D CISM_NETCDF_DIR=$NETCDF \
+  -D CMAKE_VERBOSE_MAKEFILE:BOOL=OFF \
+\
+  -D CMAKE_CXX_COMPILER=mpiicpc \
+  -D CMAKE_C_COMPILER=mpicc \
+  -D CMAKE_Fortran_COMPILER=mpif90 \
+\
+  -D CMAKE_EXE_LINKER_FLAGS="-mkl=cluster" \
+\
+  -D CMAKE_Fortran_FLAGS:STRING="-qno-opt-dynamic-align  -convert big_endian -assume byterecl -ftz -traceback -assume realloc_lhs -fp-model source -qopt-report -O2 -debug minimal " \
+  -D CMAKE_C_FLAGS:STRING="-qno-opt-dynamic-align -fp-model precise -std=gnu99 -qopt-report -O2 -debug minimal " \
+  -D CMAKE_CXX_FLAGS:STRING="-qno-opt-dynamic-align -fp-model precise -std=gnu99 -qopt-report -O2 -debug minimal " \
+ ../..
+
+# Note: last argument above  "../.."  is path to top-level cism directory

builds/derecho-intel/derecho-intel-cmake.sh

@@ -0,0 +1,73 @@
+#!/bin/sh
+
+# After this script completes, type: make -j 8
+# If rebuilding, type 'make clean' before running 'make -j 8'
+
+# Set path to top cism directory
+# Note, this is an easy way to build out of source.
+# In directory you want to build in, run:
+#   $ source $CISM/builds/linux-gnu-cism/linux-gnu-cism-cmake $CISM
+# where $CISM is the path to the top level cism directory.
+if [ $# -eq 0 ]
+then
+    cism_top="../.." 
+else
+    cism_top=${1%/}
+fi
+
+echo CISM: "${cism_top}"
+
+
+module purge
+module load ncarenv/23.06
+module load intel/2023.0.0
+module load mkl/2023.0.0
+#module load mpt/2.15f
+module load netcdf/4.9.2
+module load ncarcompilers/1.0.0
+#module load pnetcdf/1.8.0
+module load cmake/3.26.3
+#module load python/2.7.13
+#module load numpy/1.12.0
+#module load netcdf4-python/1.2.7
+
+# remove old build data:
+rm -f ./CMakeCache.txt
+rm -rf ./CMakeFiles
+
+echo
+echo "Doing CMake Configuration step"
+
+# Note: the compilation flags were taken from the defaults for a CESM build on
+# cheyenne-intel (using cime at 84aafd5). Some of these options are probably
+# unnecessary for a standalone cism build, but I am keeping things consistent
+# with the CESM build for simplicity.
+
+# CISM_USE_GPTL_INSTRUMENTATION -- ON by default, set to OFF to not use GPTL instrumentation.
+
+cmake \
+  -D CISM_BUILD_CISM_DRIVER:BOOL=ON \
+  -D CISM_ENABLE_BISICLES=OFF \
+  -D CISM_ENABLE_FELIX=OFF \
+\
+  -D CISM_USE_TRILINOS:BOOL=OFF \
+  -D CISM_MPI_MODE:BOOL=ON \
+  -D CISM_SERIAL_MODE:BOOL=OFF \
+\
+  -D CISM_USE_GPTL_INSTRUMENTATION:BOOL="${CISM_USE_GPTL_INSTRUMENTATION:=ON}" \
+  -D CISM_COUPLED:BOOL=OFF \
+  -D CISM_USE_CISM_FRONT_END:BOOL=OFF \
+\
+  -D CISM_NETCDF_DIR="$NETCDF" \
+  -D CISM_GPTL_DIR= "utils/libgptl" \
+  -D CMAKE_VERBOSE_MAKEFILE:BOOL=OFF \
+\
+  -D CMAKE_CXX_COMPILER=mpiicpc \
+  -D CMAKE_C_COMPILER=mpicc \
+  -D CMAKE_Fortran_COMPILER=mpif90 \
+\
+  -D CMAKE_Fortran_FLAGS:STRING="-qno-opt-dynamic-align  -convert big_endian -assume byterecl -ftz -traceback -assume realloc_lhs -fp-model source -qopt-report -O2 -debug minimal " \
+  -D CMAKE_C_FLAGS:STRING="-qno-opt-dynamic-align -fp-model precise -std=gnu99 -qopt-report -O2 -debug minimal " \
+  -D CMAKE_CXX_FLAGS:STRING="-qno-opt-dynamic-align -fp-model precise -std=gnu99 -qopt-report -O2 -debug minimal " \
+  "${cism_top}"
+

libglad/glad_type.F90

@@ -102,6 +102,7 @@ module glad_type
      ! Climate options -------------------------------------------
 
      integer :: evolve_ice = 1
+     logical :: zero_gcm_fluxes = .false.
 
      !> Whether the ice sheet can evolve:
      !> \begin{description}
@@ -209,6 +210,7 @@ subroutine glad_i_readconfig(instance,config)
     call GetSection(config,section,'GLAD climate')
     if (associated(section)) then
        call GetValue(section,'evolve_ice',instance%evolve_ice)
+       call GetValue(section,'zero_gcm_fluxes',instance%zero_gcm_fluxes)
        call GetValue(section,'mbal_accum_time',mbal_time_temp)
        call GetValue(section,'ice_tstep_multiply',instance%ice_tstep_multiply)
     end if
@@ -310,6 +312,11 @@ subroutine glad_i_printconfig(instance)
     if (instance%evolve_ice == EVOLVE_ICE_FALSE) then
        call write_log('The ice sheet state will not evolve after initialization')
     endif
+    if (instance%zero_gcm_fluxes) then
+       call write_log('Fluxes to the GCM will be set to zero')
+    else
+       call write_log('Fluxes to the GCM will NOT be set to zero')
+    end if
 
     if (instance%mbal_accum_time == -1) then
        call write_log('Mass-balance accumulation time will be set to max(ice timestep, mbal timestep)')

libglimmer/ncdf_template.F90.in

@@ -39,8 +39,15 @@ module NAME_io
   ! template for creating subsystem specific I/O routines
   ! written by Magnus Hagdorn, 2004
 
+  !WHL, Sept. 2023
+  ! Moved some 'use' statements to the top to avoid redundant statements that slow the build,
+  ! particularly on the Intel compiler
+
   use DATAMOD
   use glimmer_ncdf
+  use glimmer_paramets
+  use glimmer_physcon
+  use glimmer_scales
 
   implicit none
 
@@ -69,8 +76,6 @@ contains
   !*****************************************************************************
   subroutine NAME_io_createall(model,data,outfiles)
     ! open all netCDF files for output
-    use DATAMOD
-    use glide_types
     use glimmer_ncio
     implicit none
     type(glide_global_type) :: model
@@ -94,8 +99,6 @@ contains
 
   subroutine NAME_io_writeall(data,model,atend,outfiles,time)
     ! if necessary write to netCDF files
-    use DATAMOD
-    use glide_types
     use glimmer_ncio
     implicit none
     type(DATATYPE) :: data
@@ -142,14 +145,9 @@ contains
 
     use cism_parallel, only: parallel_type, &
          parallel_def_dim, parallel_inq_dimid, parallel_def_var, parallel_inq_varid, parallel_put_att
-    use glide_types
-    use DATAMOD
     use glimmer_ncio
     use glimmer_map_types
     use glimmer_log
-    use glimmer_paramets
-    use glimmer_physcon
-    use glimmer_scales
     implicit none
     type(glimmer_nc_output), pointer :: outfile
     type(glide_global_type) :: model
@@ -238,10 +236,6 @@ contains
   subroutine NAME_io_write(outfile,data)
 
     use cism_parallel, only: parallel_type, parallel_inq_varid, distributed_put_var, parallel_put_var
-    use DATAMOD
-    use glimmer_paramets
-    use glimmer_physcon
-    use glimmer_scales
     implicit none
     type(glimmer_nc_output), pointer :: outfile
     ! structure containg output netCDF descriptor
@@ -387,8 +381,6 @@ contains
   !*****************************************************************************  
   subroutine NAME_io_readall(data, model, filetype)
     ! read from netCDF file
-    use DATAMOD
-    use glide_types
     use glimmer_ncio
     implicit none
     type(DATATYPE) :: data
@@ -425,7 +417,6 @@ contains
 
     ! Read data from forcing files
     use glimmer_log
-    use glide_types
     use cism_parallel, only: main_task
 
     implicit none
@@ -520,9 +511,6 @@ contains
     use cism_parallel, only: parallel_type, &
          parallel_inq_varid, parallel_get_att, distributed_get_var, parallel_get_var
     use glimmer_log
-    use DATAMOD
-    use glimmer_paramets
-    use glimmer_scales
     implicit none
     type(glimmer_nc_input), pointer :: infile
     ! structure containg output netCDF descriptor
@@ -547,8 +535,6 @@ contains
     ! check if dimension sizes in file match dims of model
     use cism_parallel, only: parallel_type, parallel_inq_dimid, parallel_inquire_dimension
     use glimmer_log
-    use glide_types
-    use DATAMOD
     implicit none
     type(glimmer_nc_input), pointer :: infile
     ! structure containg output netCDF descriptor
@@ -575,8 +561,6 @@ contains
     ! TODO: Write code to check for doubly listed tavg variables and throw a fatal error.
 
     use cism_parallel, only: parallel_inq_varid
-    use glide_types
-    use DATAMOD
     implicit none
     type(glimmer_nc_output), pointer :: outfile
     ! structure containg output netCDF descriptor
@@ -597,7 +581,6 @@ contains
   subroutine NAME_avg_reset(outfile,data)
 
     use cism_parallel, only: parallel_inq_varid
-    use DATAMOD
     implicit none
     type(glimmer_nc_output), pointer :: outfile
     ! structure containg output netCDF descriptor

libglimmer/writestats.c

@@ -40,7 +40,7 @@
 #include <unistd.h>
 #include <string.h>
 #include <fcntl.h>
-
+#include <ctype.h>
 
 
 #define CFG_LEN 35

libglint/glint_initialise.F90

@@ -557,7 +557,7 @@ subroutine glint_i_end(instance)
     !> Tidy up 
 
     use glide
-    use glide_stop
+    use glide_stop, only : glide_finalise
     use glimmer_ncio
     implicit none
     type(glint_instance),  intent(inout) :: instance    !> The instance being initialised.

tests/MISMIP/mismip.code/README.mismip

@@ -1,12 +1,14 @@
 Instructions for setting up and running the MISMIP experiments with CISM.
 
+Note: For setting up the experiments on an NCAR computing environment, follow the steps in the README.NCAR_HPC file in the tests directory.
+
 Note: These instructions assume that you have access to the NCAR HPC Cheyenne,
 with an LIWG HPC account. If you do not have an account and would like one,
 please contact Gunter Leguy ([email protected]).
 
 See this paper for details on MISMIP:
-Pattyn,F., et al., Results of the Marine Ice Sheet Model Intercomparison Project, MISMIP, 
-The Cryosphere, 6, 573-588, doi:10.5194/tc-6-573-2012, 2012. 
+"Pattyn,F., et al., Results of the Marine Ice Sheet Model Intercomparison Project, MISMIP, 
+The Cryosphere, 6, 573-588, doi:10.5194/tc-6-573-2012, 2012."
 
 The MISMIP experiments consist of a set of advance and retreat experiments of the 
 grounding line. This is achieved by modifying the flow rate factor A. 

tests/MISMIP/mismip.code/mismipPlotGL.py

@@ -5,7 +5,7 @@
 
 
 from netCDF4 import Dataset
-from optparse import OptionParser
+from argparse import ArgumentParser
 import numpy as np
 import matplotlib.pyplot as plt
 import sys, os
@@ -194,17 +194,14 @@ def xgSemianalytic(bedType):
 
 
 # Parse options.
-optparser = OptionParser()
+parser = ArgumentParser()
 
-optparser.add_option('-x', '--expt',     dest='experiment', type='string', default = 'all', help='MISMIP experiment set to run', metavar="EXPT")
-optparser.add_option('-s', '--stat',    dest='StatChoice',type='string',default='advance', help='MISMIP experiment set to run', metavar="EXPT")
-optparser.add_option('--bed', dest='bedtopo', type='string', default ='linear',help='bed topography, linear or poly', metavar='BEDTOPO')
+parser.add_argument('-x', '--expt',     dest='experiment', type=str, default = 'all', help='MISMIP experiment set to run', metavar="EXPT")
+parser.add_argument('-s', '--stat',    dest='StatChoice',type=str,default='advance', help='MISMIP experiment set to run', metavar="EXPT")
+parser.add_argument('--bed', dest='bedtopo', type=str, default ='linear',help='bed topography, linear or poly', metavar='BEDTOPO')
 
-for option in optparser.option_list:
-    if option.default != ("NO", "DEFAULT"):
-        option.help += (" " if option.help else "") + "[default: %default]"
-options, args = optparser.parse_args()
 
+options = parser.parse_args()
 
 if options.bedtopo == 'linear':
     As             = AsLinear