Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GFSv16.3.21 - Remove WAFS from GFS #2591

Closed
KateFriedman-NOAA opened this issue May 10, 2024 · 17 comments
Closed

GFSv16.3.21 - Remove WAFS from GFS #2591

KateFriedman-NOAA opened this issue May 10, 2024 · 17 comments
Assignees
@aerorahul @KateFriedman-NOAA
Labels
production update Processing update in production

Comments

@KateFriedman-NOAA
Copy link
Member

KateFriedman-NOAA commented May 10, 2024 •
edited
Loading

Description

Remove the WAFS (formerly EMC_gfs_wafs) from the GFS, it will now be a standalone system in operations.

WAFS issue: NOAA-EMC/WAFS#23

Target version

v16.3.21 (previously v16.3.22 (previously v16.3.23))

Expected workflow changes

Remove WAFS pieces:

  • jobs in ecf def
  • ecf scripts
  • wafs job, ex-scripts, and ush scripts
  • WAFS jobs from rocoto mesh
  • configs
  • set WAFSF="NO"
  • discontinue GTG checkout and -o flag for checkout (?)
@KateFriedman-NOAA KateFriedman-NOAA added the production update Processing update in production label May 10, 2024
@KateFriedman-NOAA KateFriedman-NOAA changed the title GFSv16.4.1 - Remove WAFS from GFS GFSv16.4.0 - Remove WAFS from GFS May 10, 2024
@KateFriedman-NOAA KateFriedman-NOAA changed the title GFSv16.4.0 - Remove WAFS from GFS GFSv16.?.? - Remove WAFS from GFS Jun 10, 2024
@KateFriedman-NOAA
Copy link
Member Author

Timeline update from @aerorahul :

WAFS has a code delivery in Q1FY25 with an implementation in Q2FY25.
The GFS will be delivered and implemented at the same time removing WAFS from the GFS.

@KateFriedman-NOAA
Copy link
Member Author

KateFriedman-NOAA commented Jul 8, 2024 •
edited
Loading

This will be v16.4.0

@KateFriedman-NOAA KateFriedman-NOAA changed the title GFSv16.?.? - Remove WAFS from GFS GFSv16.4.0 - Remove WAFS from GFS Jul 8, 2024
@KateFriedman-NOAA KateFriedman-NOAA changed the title GFSv16.4.0 - Remove WAFS from GFS GFSv16.3.18 - Remove WAFS from GFS Sep 4, 2024
@KateFriedman-NOAA
Copy link
Member Author

Created branch: https://github.com/NOAA-EMC/global-workflow/tree/release/gfs.v16.3.18

@KateFriedman-NOAA KateFriedman-NOAA changed the title GFSv16.3.18 - Remove WAFS from GFS GFSv16.3.19 - Remove WAFS from GFS Sep 16, 2024
@KateFriedman-NOAA KateFriedman-NOAA mentioned this issue Sep 16, 2024
Merged
7 tasks
KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Sep 16, 2024
@KateFriedman-NOAA
Update release notes for WAFS removal from GFS
563d542 
Also update gfs_ver=v16.3.19

Refs NOAA-EMC#2591
@KateFriedman-NOAA KateFriedman-NOAA mentioned this issue Sep 16, 2024
Merged
KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Sep 18, 2024
@KateFriedman-NOAA
Update release notes to mention new file name
fc47df6 
Refs NOAA-EMC#2591
KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Sep 19, 2024
@KateFriedman-NOAA
Update release notes based on feedback from WAFS
355a6af 
Refs NOAA-EMC#2591
KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Sep 20, 2024
@KateFriedman-NOAA
Updates to file list in release notes
98b902b 
Refs NOAA-EMC#2591
KateFriedman-NOAA added a commit that referenced this issue Sep 20, 2024
@KateFriedman-NOAA
Update release notes for WAFS removal from GFS (#2927)
d661f1f 
Update the release notes for the WAFS removal from the GFS.
Files and settings that are removed are noted.

Also update `gfs_ver=v16.3.19`.

Refs #2591
@KateFriedman-NOAA
Copy link
Member Author

KateFriedman-NOAA commented Sep 24, 2024 •
edited
Loading

Just renamed the release branch to release/gfs.v16.3.21 to match the new version in ops. The ARFC for GFSv16.3.19 took the version number and the obsproc/v1.3 update will take GFSv16.3.20. Will update release notes and internal version shortly, need to prep release branch for ARFC GFSv16.3.19 first.

@RussTreadon-NOAA
Copy link
Contributor

Thank you @KateFriedman-NOAA for your quick action.

DA team members are meeting this afternoon to make a final decision as to what course of action to take. Submitting an ARFC to place PlanetiQ data in monitor mode is one option. Another option is to let the PlanetiQ data flow into and be assimilated by the GFS. This later option requires no EIB action.

This issue is opened as a "heads up" to EIB in case the DA team goes with the first option.

@KateFriedman-NOAA KateFriedman-NOAA mentioned this issue Sep 24, 2024
Closed
10 tasks
@KateFriedman-NOAA KateFriedman-NOAA changed the title GFSv16.3.19 - Remove WAFS from GFS GFSv16.3.21 - Remove WAFS from GFS Sep 24, 2024
KateFriedman-NOAA added a commit that referenced this issue Oct 1, 2024
@KateFriedman-NOAA
Update WAFS branch to v16.3.21
c019ab5 
Refs #2591
@KateFriedman-NOAA
Copy link
Member Author

Synced the release/gfs.v16.3.21 branch with dev/gfs.v16 after the GFSv16.3.19 branch was merged into dev/gfs.v16. Also updated the release branch for the new version v16.3.21.

KateFriedman-NOAA added a commit that referenced this issue Oct 9, 2024
@KateFriedman-NOAA
Sync merge GFSv16.3.20 changes into GFSv16.3.21 release branch
56a2243 
Merge remote-tracking branch 'origin/release/gfs.v16.3.20' into release/gfs.v16.3.21

Refs #2591

* origin/release/gfs.v16.3.20:
  Update GSI tag to gfsda.v16.3.20 for saildrone data (#2986)
  Update WAFS tag in release notes
  Set WAFS tag back to new gfs_wafs.v6.3.3
  Revert "Remove WAFS from ops (#2889)"
  Update for obsproc/v1.3 and new AFWA global snow filename (#2957)
@KateFriedman-NOAA
Copy link
Member Author

Sync merged the recently-handed-off changes for GFSv16.3.20 into the release/gfs.v16.3.21 branch and cut hand-off tag EMC-v16.3.21.

@KateFriedman-NOAA
Copy link
Member Author

CDF submitted to NCO.

@KateFriedman-NOAA
Copy link
Member Author

WAFS CDF was also just submitted to NCO. EVS will also be submitting a companion CDF soon.

@KateFriedman-NOAA
Copy link
Member Author

@YaliMao-NOAA NCO has requested that we increase the walltime of the wafs 0p25 blending job to 40mins in the new WAFSv7.0.0 package. NCO made the change locally in the GFSv16.3.20 install:

I increased the walltime of 
/lfs/h1/ops/para/packages/gfs.v16.3.20/ecf/scripts/gfs/atmos/post_processing/grib2_wafs/jgfs_atmos_wafs_blending_0p25.ecf
from 30 to 40 minutes to match a change Wei made in gfs.v16.3.18. - Diane Stokes

Please make the same increase in the WAFSv7.0.0 package and recut the WAFS hand-off tag. Thanks!

@YaliMao-NOAA
Copy link
Contributor

YaliMao-NOAA commented Oct 11, 2024 •
edited
Loading

@KateFriedman-NOAA Thank you for the notification.
I am going to check whether it's necessary to increase the walltime in wafsv7.0.0 since blending process is parallel in wafsv7.0.0

@KateFriedman-NOAA KateFriedman-NOAA mentioned this issue Oct 30, 2024
Merged
KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Oct 30, 2024
@KateFriedman-NOAA
Add enkfgdas_update resource/env change to release notes
3e23ead 
Refs NOAA-EMC#2591
KateFriedman-NOAA added a commit that referenced this issue Oct 30, 2024
@KateFriedman-NOAA
Update enkfgdas_update job resource/env settings (#3047)
ab8e453 
Updates for the `enkfgdas_update` job from testing by NCO/GDIT and EMC.
They produce a more stable runtime and bitwise identical outputs to ops.

Add `export OMP_PROC_BIND=true` and update `APRUN_ENKF` to use
`--cpu-bind core` instead of `--cpu-bind depth` in WCOSS2.env. 
Also, adjust npe (280) and thread (16) values.

NCO requests that we include these updates in the GFSv16.3.21 package.

Refs #2591
KateFriedman-NOAA added a commit that referenced this issue Oct 30, 2024
@KateFriedman-NOAA
Merge remote-tracking branch 'origin/release/gfs.v16.3.20' into relea…
bc1dd7d 
…se/gfs.v16.3.21

Refs #2591

* origin/release/gfs.v16.3.20:
  Add sfcprep to config.resources.nco.static (#3031)
@KateFriedman-NOAA
Copy link
Member Author

Merge synced the release/gfs.v16.3.20 changes into the release/gfs.v16.3.21 branch and recut the EMC-v16.3.21 tag.

@KateFriedman-NOAA KateFriedman-NOAA changed the title GFSv16.3.21 - Remove WAFS from GFS GFSv16.3.22 - Remove WAFS from GFS Nov 8, 2024
@KateFriedman-NOAA
Copy link
Member Author

This is now v16.3.22. The new ops change documented in issue #3082 will go in before this and pushes this to v16.3.22.

KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Nov 13, 2024
@KateFriedman-NOAA
Update GFS version to v16.3.22
0de673d 
Refs NOAA-EMC#2591
@KateFriedman-NOAA KateFriedman-NOAA mentioned this issue Nov 13, 2024
Merged
KateFriedman-NOAA added a commit that referenced this issue Nov 13, 2024
@KateFriedman-NOAA
Update GFS version to v16.3.22 (#3095)
9861ce0 
Update the version for the "WAFS removal from GFS" release branch to v16.3.22.

Refs #2591
@KateFriedman-NOAA KateFriedman-NOAA changed the title GFSv16.3.22 - Remove WAFS from GFS GFSv16.3.23 - Remove WAFS from GFS Dec 9, 2024
@KateFriedman-NOAA KateFriedman-NOAA changed the title GFSv16.3.23 - Remove WAFS from GFS GFSv16.3.22 - Remove WAFS from GFS Dec 18, 2024
KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Dec 18, 2024
@KateFriedman-NOAA
Remake the WAFS removal from GFS package
eca3771 
- updated release notes
- remove all WAFS scripts and settings
- include resource update for enkfgdas_update job

Refs NOAA-EMC#2591
@KateFriedman-NOAA KateFriedman-NOAA mentioned this issue Dec 18, 2024
Merged
KateFriedman-NOAA added a commit that referenced this issue Dec 18, 2024
@KateFriedman-NOAA
Remake the package for WAFS removal from GFS (#3176)
22a7008 
Remake the release branch for the WAFS removal from the GFS due
to implementation order change. It includes changes previously made in
the original release branch:

- updated release notes
- remove all WAFS scripts and settings
- include resource update for enkfgdas_update job

Refs #2591
KateFriedman-NOAA pushed a commit that referenced this issue Dec 18, 2024
@YaliMao-NOAA
Updated release note related to WAFS separation changes (#3179)
3ad6fe1 
Refs #2591
@KateFriedman-NOAA KateFriedman-NOAA changed the title GFSv16.3.22 - Remove WAFS from GFS GFSv16.3.21 - Remove WAFS from GFS Dec 19, 2024
KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Jan 3, 2025
@KateFriedman-NOAA
Update for syndat_getjtbul code
6705417 
The gfs_atmos_tropcy_qc_reloc did not properly parse the vitals info from a raw bulletin
with a tab in the storm name field for storm "SIX". The code sorc/syndat_getjtbul.fd/getjtbul.d
converting a tab to a space worked well for storms that have at least 4 characters in their names,
but does not work as well for storms with 3-character names. This code change will fix the
parsing problem for 3-character storms.

Refs NOAA-EMC#2591
@KateFriedman-NOAA KateFriedman-NOAA mentioned this issue Jan 3, 2025
Merged
3 tasks
KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Jan 6, 2025
@KateFriedman-NOAA
Update comments related to number of year digits
1cee9a0 
Refs NOAA-EMC#2591
KateFriedman-NOAA added a commit that referenced this issue Jan 6, 2025
@KateFriedman-NOAA
Fix for syndat_getjtbul code parsing 3-character storm names (GFSv16.…
65f2645 
…3.21) (#3200)

Update the `sorc/syndat_getjtbul.fd/getjtbul.f` code to resolve a problem parsing 3-character storm names.
This update is being included in the GFSv16.3.21 package (with approval by NCO).

From the developer @Qingfu-Liu:

> The `gfs_atmos_tropcy_qc_reloc` job did not properly parse the vitals
info from a raw bulletin with a tab in the storm name field for storm
"SIX". The code sorc/syndat_getjtbul.fd/getjtbul.f converting a tab to a
space worked well for storms that have at least 4 characters in their
names, but does not work as well for storms with 3-character names. This
code change will fix the parsing problem for 3-character storms.

Refs #2591
@KateFriedman-NOAA
Copy link
Member Author

Update from NCO/Xiaoxue:

WAFS implementation is delayed to 1/29 per AWC's request. The 30-day is for WAFS was completed on 1/15 (started on 12/17).

@KateFriedman-NOAA KateFriedman-NOAA mentioned this issue Jan 24, 2025
Closed
@KateFriedman-NOAA
Copy link
Member Author

From the January 24th 2025 RFC memo:

RFC 13621 - On WCOSS2, implement WAFS (World Area Forecast System)
v7.0.1 into operations by removing the evaluation flag and turning on production
DBnet alerts. This implementation is a separation of the WAFS component from
the GFSv16 application. There is no science change. This will allow for easier
future updates of the WAFS since it will no longer be embedded in the GFS. To
be implemented on January 29 from 1430Z to 2000Z and on January 30 from
1400Z to 1500Z.

@KateFriedman-NOAA KateFriedman-NOAA mentioned this issue Jan 27, 2025
Merged
aerorahul pushed a commit that referenced this issue Jan 27, 2025
@KateFriedman-NOAA
Remove WAFS files and references from develop (#3263)
76cecfb 
This PR removes all remaining WAFS files and references from
global-workflow `develop`. The WAFS is being separated from the GFS and
becoming its own package in operations. See #2591.

Resolves #3253
@KateFriedman-NOAA KateFriedman-NOAA mentioned this issue Jan 29, 2025
Merged
KateFriedman-NOAA added a commit that referenced this issue Jan 29, 2025
@KateFriedman-NOAA
GFSv16.3.21 - WAFS removal from GFS (#3281)
132a2c4 
The WAFS is now a separate package in operations and is no longer a part of the GFS.

Also included in this package are the following unrelated updates:

1. improvements to the enkfgdas_update job resource and environment settings
2. a fix for the syndat_getjtbul code when parsing 3-character storm names

Refs #2591
@KateFriedman-NOAA
Copy link
Member Author

Tagged and released: https://github.com/NOAA-EMC/global-workflow/releases/tag/gfs.v16.3.21

Announced to users. Closing this issue as complete. Thanks all!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@aerorahul aerorahul

@KateFriedman-NOAA KateFriedman-NOAA

Labels
production update Processing update in production
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@aerorahul @RussTreadon-NOAA @KateFriedman-NOAA @YaliMao-NOAA