Plotting stats task bug

[EdwardSnyder-NOAA]

While testing the release v2.0.0 land da container I noticed this error message for the plot_stats task:

File date= 2000-01-03-00
File date= 2000-01-03-00
Traceback (most recent call last):
  File "/gpfs/f5/epic/scratch/Edward.Snyder/landda-cont/release-v2/land-DA_workflow/ush/plot_analysis_timehistory.py", line 293, in <module>
    main()
  File "/gpfs/f5/epic/scratch/Edward.Snyder/landda-cont/release-v2/land-DA_workflow/ush/plot_analysis_timehistory.py", line 53, in main
    var_dict_anal = get_data_analysis(path_data,fn_data_anal_prefix,fn_data_anal_suffix,nprocs_anal,var_nm)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/gpfs/f5/epic/scratch/Edward.Snyder/landda-cont/release-v2/land-DA_workflow/ush/plot_analysis_timehistory.py", line 137, in get_data_analysis
    min_val_final.append(min_val_file[-1])
                         ~~~~~~~~~~~~^^^^
IndexError: list index out of range

After placing print statements in the loop of the plot_analysis_timehistory.py script, I realized that the files variable is containing more than one analysis log file, which in my case where error log files as in ptmp/test/com/output/logs/analysis_2000010300.log.*. The work around was to delete these extra log files in the the logs directory. After that, the plot_stats task completed as expected. What is strange is the plot_timehistory.yaml is calling the '.log' extension explicitly: fn_data_fcst_suffix: '.log'. So I'm not sure what is going on here exactly.

[chan-hoo]

@EdwardSnyder-NOAA, I think this might be caused by that the log file (analsys_xxx.log) was being changed when the plot_stat task ran for some reason. Each task has maxtries=2 in the rocoto workflow. So it might work well in the next try. Did the ploat_stat fail with DEAD finally? If so, I'll try to reproduce this kind of error and provide a solution.

[chan-hoo]

@EdwardSnyder-NOAA, I think this part (

land-DA_workflow/ush/plot_analysis_timehistory.py

Line 67 in 16c62fb

if entry.is_file() and entry.name.startswith(fn_data_anal_prefix):

) has a potential issue as you mentioned. I'll add another condition using endswith and let you know what happens.