Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/ETC_composites #153

Open
wants to merge 135 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 67 commits
Commits
Show all changes
135 commits
Select commit Hold shift + click to select a range
dc247e4
added feature/eulerian-storm-track by Jeyavinoth Jeyaratnam
jeyavinoth Oct 29, 2019
a79ad0a
merged MDTF develop branch for my eulerian storm tracker
jeyavinoth Jan 17, 2020
cd20158
ready to work on eulerian-storm-tracker
jeyavinoth Jan 17, 2020
d4c8403
changing settings.json for my eulerian-strom-track
jeyavinoth Jan 17, 2020
7a1d6d7
copying over new code
jeyavinoth Jan 17, 2020
1a73831
working version of eulerian_storm_tracker; needs more finer changes
jeyavinoth Jan 17, 2020
dfa152c
adding wheeler kiladis to my test run; index.html has a bug when only…
jeyavinoth Jan 17, 2020
8edcffb
Merge branch 'feature/eulerian-storm-track' into develop
jeyavinoth Jan 18, 2020
4236606
upstream update of develop
jeyavinoth Aug 21, 2020
b53a427
Merge remote-tracking branch 'upstream/develop' into develop
jeyavinoth Sep 3, 2020
7e54769
trying out eulerian_storm_track
jeyavinoth Sep 3, 2020
ac27821
eulerian storm track works, but now have to filter out the topography…
jeyavinoth Sep 20, 2020
3c47b35
create topo file for obs data
jeyavinoth Sep 20, 2020
5a3281f
added zonal means
jeyavinoth Sep 22, 2020
c6acb99
added labels
jeyavinoth Sep 23, 2020
fe98178
cleaned the eulerian_storm_track.yml file
jeyavinoth Sep 23, 2020
4c6c392
copied over the tracker info the framework
jeyavinoth Sep 23, 2020
0d463d5
started working on the etc-composites
jeyavinoth Sep 24, 2020
c435373
updated the pdf
jeyavinoth Sep 24, 2020
5b6e81e
removed old front_detection codde
jeyavinoth Sep 24, 2020
e5ff616
removed old front_detection codde
jeyavinoth Sep 24, 2020
225386b
manually editted the yml env file
jeyavinoth Sep 24, 2020
ef59c09
updated the documentation
jeyavinoth Sep 24, 2020
ac20eeb
created yml file for etc_composites
jeyavinoth Sep 24, 2020
ddd5761
added back the etc_composites stuff
jeyavinoth Sep 24, 2020
5de80bf
the center finder runs fine, but crashes on track finder cuz of year …
jeyavinoth Oct 5, 2020
3bd376d
working version of the tracker part; it creates the .mat file
jeyavinoth Oct 6, 2020
3f313f5
etc_composites done up to track_stats
jeyavinoth Oct 6, 2020
0b70d37
created the composites images; have to add it to the html
jeyavinoth Oct 7, 2020
083d6f0
removed eulerian_storm_track from git repo
jeyavinoth Oct 7, 2020
1a1346c
added back the eulerian storm track
jeyavinoth Oct 7, 2020
e7f69cf
Merge branch 'develop' of https://github.com/NOAA-GFDL/MDTF-diagnosti…
jeyavinoth Oct 8, 2020
4374e27
updated the html file to include the etc_composites images
jeyavinoth Oct 8, 2020
13e115f
fixed an error where the year in the dir path was getting replaced; a…
jeyavinoth Oct 8, 2020
67089a4
editted etc_composites.html; have to decide on the final layout
jeyavinoth Oct 8, 2020
2d080bb
added obs and reanalysis comparison
jeyavinoth Oct 20, 2020
aa943fe
made changes to the html page
jeyavinoth Oct 21, 2020
67ed051
trying to create the difference plot for the composites
jeyavinoth Oct 23, 2020
e707f9e
composites are created and difference plots also created
jeyavinoth Oct 29, 2020
50b13a9
tracker is run based on a flag in settings.jsonc
jeyavinoth Nov 1, 2020
c418c42
composites code running on different tracker output - prelim done
jeyavinoth Nov 4, 2020
f696100
creating plots for AGU
jeyavinoth Nov 16, 2020
fd76109
agu plotting
jeyavinoth Nov 16, 2020
af871d0
completed agu plots for eulerian plots
jeyavinoth Nov 16, 2020
ef73b01
saved track_stats.pkl to wk_dir folder; updated agu plots
jeyavinoth Nov 16, 2020
964db13
created separate input folder for leoc96; creating inputdata for 2pm
jeyavinoth Nov 19, 2020
9f50056
successfully ran era-interim for 2008 to2012; erai conversion code
jeyavinoth Dec 10, 2020
6bec76d
testing era5 run
jeyavinoth Dec 11, 2020
2ccaa10
successfully ran era5; figures looks good
jeyavinoth Dec 15, 2020
dce07e8
added a flag to run composites or not in settings.jsonc
jeyavinoth Jan 5, 2021
963d79d
added compilation of Cython files
jeyavinoth Jan 6, 2021
5c9a0e5
the pod now works when run without compositing
jeyavinoth Jan 7, 2021
ad3a617
running era5 all; fixed html
jeyavinoth Jan 11, 2021
51365ab
successfully ran ERA5 for all years; create empty _vars plots
jeyavinoth Jan 12, 2021
0da8bc5
created new diff plots; fixed some minor errors
jeyavinoth Jan 12, 2021
8e06cb4
lat_distrib; creating new flags for mat files
jeyavinoth Jan 31, 2021
022a69e
match the latitude distribution to the observation
jeyavinoth Feb 1, 2021
be3f2a6
fixed an issue with converting txt tracks to mat files
jeyavinoth Feb 9, 2021
473c091
changed the landsea threshold to 0 from 50
jeyavinoth Feb 9, 2021
3920727
getting ready to push to noaa github; have to clean up
jeyavinoth Feb 24, 2021
dda576a
Merge branch 'develop' of https://github.com/NOAA-GFDL/MDTF-diagnosti…
jeyavinoth Feb 25, 2021
79e95be
Merge branch 'develop' into feature/ETC-composites
jeyavinoth Feb 25, 2021
9ff1fff
merged with upstream/develop
jeyavinoth Feb 25, 2021
1771086
cleaning up code for PR
jeyavinoth Mar 1, 2021
b324dcd
Merge branch 'develop' of https://github.com/NOAA-GFDL/MDTF-diagnosti…
jeyavinoth Mar 1, 2021
821d9d1
Merge branch 'develop' into feature/ETC-composites
jeyavinoth Mar 1, 2021
7f7ade9
merge with develop not running my pod
jeyavinoth Mar 3, 2021
1ce32f3
Merge branch 'develop' of https://github.com/NOAA-GFDL/MDTF-diagnosti…
jeyavinoth Mar 3, 2021
935f039
Merge branch 'develop' into feature/ETC-composites
jeyavinoth Mar 3, 2021
79ab478
committing my code before merging an update from NOAA that accounts f…
jeyavinoth May 7, 2021
84193a8
Merge remote-tracking branch 'upstream/develop' into develop
jeyavinoth May 7, 2021
0c3cfce
Merge branch 'develop' into feature/ETC-composites
jeyavinoth May 7, 2021
ec838de
tried getting static files to work; but failed
jeyavinoth May 21, 2021
4b67891
Merge branch 'develop' of https://github.com/NOAA-GFDL/MDTF-diagnosti…
jeyavinoth May 21, 2021
4039d3a
included variable selection; minor html changes
jeyavinoth Jul 12, 2021
b4cfcd5
included variable selection; minor html changes
jeyavinoth Jul 12, 2021
b6121a1
Merge remote-tracking branch 'upstream/develop' into feature/ETC-comp…
jeyavinoth Jul 12, 2021
7c8203d
minor fixes (fixed variable confusion; tp->pr,w500->w500)
jeyavinoth Jul 12, 2021
5b190a0
fixed an issue with thresh_landsea; now separated as landsea_hgt/lsm
jeyavinoth Aug 4, 2021
eecf184
add CM4 diag table to repo (#269)
wrongkindofdoctor Aug 23, 2021
6aa5764
Improve mamba usage in CI
tsjackson-noaa Aug 27, 2021
882fa08
Merge pull request #272 from tsjackson-noaa/feature/mamba-CI
tsjackson-noaa Aug 27, 2021
92d9551
Add temp extremes distshape to CI (#273)
wrongkindofdoctor Sep 9, 2021
b8d2332
Indicate which settings are required, in website and CLI help
tsjackson-noaa Sep 16, 2021
6894215
Move --large-file flag higher in list of options, so that data_manage…
tsjackson-noaa Sep 16, 2021
43e7525
Clarify behavior of --site and plug-in settings
tsjackson-noaa Sep 16, 2021
68efcc0
Don't do full startup when only printing CLI help message
tsjackson-noaa Sep 16, 2021
e6013f2
Apply changes to CLI to NOAA_GFDL site
tsjackson-noaa Sep 16, 2021
19b5206
Correct GFDL default paths to values in default_gfdl.jsonc
tsjackson-noaa Sep 16, 2021
2597160
Do more validation on input paths
tsjackson-noaa Sep 16, 2021
c9f6ab3
Do more input path validation in GFDL-specific code
tsjackson-noaa Sep 16, 2021
539bf02
Validate CASE_ROOT_DIR before other checks
tsjackson-noaa Sep 16, 2021
009dd89
Only validate non-empty CASE_ROOT_DIR
tsjackson-noaa Sep 16, 2021
acc56b6
Merge pull request #276 from tsjackson-noaa/feature/cli-docs
tsjackson-noaa Sep 17, 2021
f7ab309
Remove Tom from codeowners file
wrongkindofdoctor Sep 28, 2021
9eec807
Merge of Tropical Sea Level POD (#271)
jkrasting Oct 4, 2021
d382974
Add surface variable support (#266)
wrongkindofdoctor Oct 25, 2021
f8f3e3c
add 3D and 4D wind_speed fields to CMIP fieldlist (#285)
wrongkindofdoctor Oct 25, 2021
d37ba3a
Add 3D specific humidity to CMIP fieldlist (#286)
wrongkindofdoctor Oct 25, 2021
1923382
Fix FRE integration and improve documentation (#245)
tsjackson-noaa Oct 26, 2021
daac4e4
change 3d wind_speed to sfcWind in CMIP fieldlist (#288)
wrongkindofdoctor Oct 28, 2021
f36cb66
Document internal APIs (#274)
tsjackson-noaa Oct 28, 2021
60632d3
update env_base.yml (#289)
wrongkindofdoctor Oct 28, 2021
68eea42
Revert "update env_base.yml (#289)" (#290)
wrongkindofdoctor Oct 28, 2021
55f75d9
Change cf_time calendar logic (#291)
wrongkindofdoctor Nov 4, 2021
03c60fe
Add metpy to python3_base yaml (#294)
wrongkindofdoctor Nov 15, 2021
3cae945
add test yaml for trop pac sl pod (#281)
wrongkindofdoctor Nov 16, 2021
54b4593
Merge branch 'main' into develop
wrongkindofdoctor Nov 16, 2021
8e2c804
Add base yaml pkgs (#297)
wrongkindofdoctor Nov 19, 2021
589eca2
Mld debug (#283)
wrongkindofdoctor Nov 30, 2021
6600a62
Merge branch 'main' into develop
wrongkindofdoctor Dec 1, 2021
e0811f1
Enso mse (#292)
jhafner2 Dec 2, 2021
0b018a6
Feature/ocean flux matrix (New POD) (#293)
chiaweh2 Dec 2, 2021
5b7d3a1
Enso rws (#223)
jhafner2 Dec 2, 2021
64b373f
change developer cheatsheet link
wrongkindofdoctor Dec 7, 2021
3172cff
Feature/ocean flux matrix (merged POD update on rst) (#300)
chiaweh2 Dec 13, 2021
0d24321
Add pods to ci (#301)
wrongkindofdoctor Dec 16, 2021
858303d
Merge branch 'main' into develop
wrongkindofdoctor Dec 16, 2021
ce68aef
Merge remote-tracking branch 'upstream/develop' into feature/ETC-comp…
jeyavinoth Dec 17, 2021
4e36b5b
Merge remote-tracking branch 'upstream/main' into feature/ETC-composites
jeyavinoth Dec 18, 2021
b41168c
Merge branch 'NOAA-GFDL:main' into feature/ETC-composites
jeyavinoth Dec 18, 2021
1e7620c
Merge remote-tracking branch 'upstream/main' into feature/ETC-composites
jeyavinoth Mar 1, 2022
0443db3
Fix Explicit_file CLI definition
wrongkindofdoctor Mar 1, 2022
7eb04ad
Update the CM4 diag table
wrongkindofdoctor Mar 3, 2022
9da4905
remove --disable-epsv option from curl calls
wrongkindofdoctor Mar 4, 2022
748d6da
Merge branch 'main' into develop
wrongkindofdoctor Mar 7, 2022
87316ea
updated POD to get it working with the new varlist
jeyavinoth Mar 15, 2022
d0f8831
Merge branch 'feature/ETC-composites' of github.com:jeyavinoth/MDTF-d…
jeyavinoth Mar 16, 2022
5cbaaf3
Merge remote-tracking branch 'upstream/develop' into feature/ETC-comp…
jeyavinoth Mar 16, 2022
5718729
cleaned up most of the unnecessary files
jeyavinoth Mar 16, 2022
8697263
cleaned up more files; remove some lgtm alerts
jeyavinoth Mar 16, 2022
09c3bbd
removed py files not used in tracker
jeyavinoth Mar 20, 2022
1f1b2b5
modified files to take care of lgtm alerts
jeyavinoth Mar 20, 2022
6d5460c
removed make_scree_v4.py
jeyavinoth Mar 20, 2022
7f9732c
cleaned up some files in original tracker
jeyavinoth Mar 30, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file not shown.
181 changes: 181 additions & 0 deletions diagnostics/etc_composites/etc_composites.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
<!-- This file is part of the convective_transition_diag module of the MDTF code package (see mdtf/MDTF_v2.0/LICENSE.txt) -->

<STYLE>
table, th, td {border: 1px solid black;}
</STYLE>

<IMG SRC="../mdtf_diag_banner.png" border=0 usemap="#logo">
<TITLE>ETC Composites</TITLE>
<H3>ETC Composite Analysis</h3>
<p>
The Extratropical cyclones are identified with the Modeling, Analysis
and Prediction (MAP) Climatology of Midlatitude
Storminess (MCMS) algorithm (Bauer et al. 2016)
which uses 6-hourly gridded sea level pressure fields to
locate storm centers and then track them through the
cyclone lifetime. This algorithm has historically been
used to create a database of cyclone locations using the
ERA-Interim reanalysis (Dee et al. 2011). The MCMS algorithm
is also applied to the model and re-analysis
sea level pressure fields.

We generate cyclone centered composites of cloud cover,
as well as other
atmospheric variables, over multiple instances of the
same type of atmospheric phenomenon to compare the
models with observations or reanalysis.
The is a plan view as a passive
instrument would observe.
This method presents a
great advantage for model evaluation as they allow
multiple cases to be included and do not necessitate a
match in time and space between the free running
models and observations.

<p>
<a href="MDTF_Documentation_etc_composites.pdf">Full Documentation and Contact Information</a>
<P>
<P>
<H3>Track Stats: </H3> <A HREF=model/{{CASENAME}}_track_stats.png target="_blank">{{CASENAME}}</A>
<P>
<P>
<H3>Variable Legend</H3>
<P>
<TABLE>
<TR>
<TH>Variable</TH>
<TH>Long Name</TH>
<TH>Units</TH>
</TR>
<TR>
<TD>PR</TD>
<TD>Total Precipitation</TD>
<TD>mm/hr</TD>
</TR>
<TR>
<TD>PRW</TD>
<TD>Total Column Water Vapor</TD>
<TD>mm/hr</TD>
</TR>
<TR>
<TD>UV10</TD>
<TD>10m Wind Speeds [sqrt(U^2 + V^2)]</TD>
<TD>m/s</TD>
</TR>
<TR>
<TD>W500</TD>
<TD>Vertical Velocity @ 500 hPA</TD>
<TD>Pa/s</TD>
</TR>
<TR>
<TD>CLT</TD>
<TD>Total Cloud Fraction</TD>
<TD>%</TD>
</TR>
</TABLE>

<P>
<H3>ETC Composites</H3>
<P>


<!-- Figures -->
<H3>Difference Plots </H3>
<TABLE>
<TR>
<TH>{{CASENAME}} - ERA-Interim</TH>
<TH>
<A HREF=model/diff_{{CASENAME}}_erai_vars_SH_ocean_WARM.png target="_blank">SH</A> .
<A HREF=model/diff_{{CASENAME}}_erai_vars_NH_ocean_WARM.png target="_blank">NH</A>
</TH>
</TR>
<TR>
<TH>{{CASENAME}} - MERRA</TH>
<TH>
<A HREF=model/diff_{{CASENAME}}_merra_vars_SH_ocean_WARM.png target="_blank">SH</A>
</TH>
</TR>
<TR>
<TH>{{CASENAME}} - MODIS</TH>
<TH>
<A HREF=model/diff_{{CASENAME}}_modis_vars_SH_ocean_WARM.png target="_blank">SH</A>
</TH>
</TR>
</TABLE>


<P>
<TABLE width=100%>
<TR>
<TH>Hemis - Land/Ocean - Season</TH>
<TH>{{CASENAME}}</TH>
<TH>ERA-Interim (2008-2012)</TH>
<TH>MERRA (2008-2012)</TH>
<TH>Observations (2008-2012)</TH>
</TR>

<TR>
<TH></TH>
<TH></TH>
<TH></TH>
<TH></TH>
<TH></TH>
</TR>
<H3>Individual Composites</H3>
<TR>
<TH>SH - Ocean - WARM</TH>
<TH>
<A HREF=model/{{CASENAME}}_area_tp_SH_ocean_WARM.png target="_blank">PR</A> .
<A HREF=model/{{CASENAME}}_area_prw_SH_ocean_WARM.png target="_blank">PRW</A> .
<A HREF=model/{{CASENAME}}_area_uv10_SH_ocean_WARM.png target="_blank">UV10</A> .
<A HREF=model/{{CASENAME}}_area_w500_SH_ocean_WARM.png target="_blank">W500</A> .
<A HREF=model/{{CASENAME}}_area_clt_SH_ocean_WARM.png target="_blank">CLT</A>
</TH>
<TH>
<A HREF=obs/{{CASENAME}}_erai_tp_SH_ocean_WARM.png target="_blank">PR</A> .
<A HREF=obs/{{CASENAME}}_erai_prw_SH_ocean_WARM.png target="_blank">PRW</A> .
<A HREF=obs/{{CASENAME}}_erai_uv10_SH_ocean_WARM.png target="_blank">UV10</A> .
<A HREF=obs/{{CASENAME}}_erai_w500_SH_ocean_WARM.png target="_blank">W500</A>
</TH>
<TH>
<A HREF=obs/{{CASENAME}}_merra_prw_SH_ocean_WARM.png target="_blank">PRW</A> .
<A HREF=obs/{{CASENAME}}_merra_w500_SH_ocean_WARM.png target="_blank">W500</A>
</TH>
<TH>
<A HREF=obs/{{CASENAME}}_modis_cld_SH_ocean_WARM.png target="_blank">CLT</A>
</TH>
</TR>

<TR>
<TH>NH - Ocean - WARM</TH>
<TH>
<A HREF=model/{{CASENAME}}_area_tp_NH_ocean_WARM.png target="_blank">PR</A> .
<A HREF=model/{{CASENAME}}_area_prw_NH_ocean_WARM.png target="_blank">PRW</A> .
<A HREF=model/{{CASENAME}}_area_uv10_NH_ocean_WARM.png target="_blank">UV10</A> .
<A HREF=model/{{CASENAME}}_area_w500_NH_ocean_WARM.png target="_blank">W500</A> .
<A HREF=model/{{CASENAME}}_area_clt_NH_ocean_WARM.png target="_blank">CLT</A>

</TH>
<TH>
<A HREF=obs/{{CASENAME}}_erai_tp_NH_ocean_WARM.png target="_blank">PR</A> .
<A HREF=obs/{{CASENAME}}_erai_prw_NH_ocean_WARM.png target="_blank">PRW</A> .
<A HREF=obs/{{CASENAME}}_erai_uv10_NH_ocean_WARM.png target="_blank">UV10</A> .
<A HREF=obs/{{CASENAME}}_erai_w500_NH_ocean_WARM.png target="_blank">W500</A>
</TH>
<TH>
</TH>
<TH>
</TH>
</TR>

<TR>
<TH></TH>
<TH></TH>
<TH></TH>
<TH></TH>
<TH></TH>
</TR>
</TABLE>

<P>

Loading