Forecast output with sub-hour interval

[byoung-joo]

Potential suggestions

Users can control the output interval with outIntervalHR key for the extended forecast capability. For example, see https://github.com/NCAR/MPAS-Workflow/blob/develop/scenarios/3dhybrid_O30kmIE60km_SpecifiedEnsemble_VarBC_allConfig.yaml#L414 .

I was trying to write out the forecast output with 30 minutes interval, so set outIntervalHR: 0.5. However, this has failed with some error.

It might be useful if we can make this key work with sub-hour interval.

This will require some modification on the scripts that populates the MPAS configuration files.

[byoung-joo]

Additional note:

If we want to do somehow the usual obs space verification on the extended forecasts of sub-hour interval, this will require the hofx application working with sub-hour interval. Eventually we may want to consider sub-hour time control in the MPAS-Workflow, which is currently configurable as hourly.

@st-ncar (hourly cycling) or @ibanos90 (abi/ahi da/verif), please feel free to add your thoughts.

[st-ncar]

Is the error message you mention thrown by cylc? I wonder if cylc supports floating point values for datetime recurrences in the workflow.

[byoung-joo]

Is the error message you mention thrown by cylc? I wonder if cylc supports floating point values for datetime recurrences in the workflow.

Forgot the details, but it was from the script level error trap (python?). By setting outIntervalHR: 0.5, the forecast length became effectively 0, then gave the error. If useful, I can reproduce it and add here!

[byoung-joo]

So, when I set outIntervalHR: 0.5, which is defined as integer (https://github.com/NCAR/MPAS-Workflow/blob/develop/initialize/applications/ExtendedForecast.py#L36), I got the following error.

#########################################################################
Running the scenario: xxx.yaml
#########################################################################
Traceback (most recent call last):
  File "/glade/work/bjung/panda-c/workflow/MPAS-workflow.20260112_for_ML_cmpr/./Run.py", line 108, in <module>
    if __name__ == '__main__': main()
                               ^^^^^^
  File "/glade/work/bjung/panda-c/workflow/MPAS-workflow.20260112_for_ML_cmpr/./Run.py", line 44, in main
    run.execute()
  File "/glade/work/bjung/panda-c/workflow/MPAS-workflow.20260112_for_ML_cmpr/./Run.py", line 90, in execute
    suite = SuiteLookup(suiteName, scenario.getConfig())
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/glade/work/bjung/panda-c/workflow/MPAS-workflow.20260112_for_ML_cmpr/initialize/suites/SuiteBase.py", line 206, in SuiteLookup
    return suiteDict[suiteName](conf)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/glade/work/bjung/panda-c/workflow/MPAS-workflow.20260112_for_ML_cmpr/initialize/suites/ForecastFromExternalAnalyses.py", line 48, in __init__
    self.c['extendedforecast'] = ExtendedForecast(conf, self.c['hpc'], self.c['members'], self.c['forecast'],
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/glade/work/bjung/panda-c/workflow/MPAS-workflow.20260112_for_ML_cmpr/initialize/applications/ExtendedForecast.py", line 93, in __init__
    extLengths = range(0, lengthHR+outIntervalHR, outIntervalHR)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: range() arg 3 must not be zero

With a quick look from https://cylc.github.io/cylc-doc/stable/html/tutorial/scheduling/datetime-cycling.html#iso8601-datetimes , minute and seconds seem to be available from cylc, but I might misunderstand.

[st-ncar]

Thanks for posting the output. You are right, this error is not related to cylc. I think this is a type conversion issue instead. outIntervalHR is cast to type integer in this line:

MPAS-Workflow/initialize/config/Config.py

Line 83 in ebe511b

return typ(v)

This conversion triggers the above error because int(0.5) = 0.

Perhaps you can get it to work by declaring
'outIntervalHR': [12.0, float],
in

MPAS-Workflow/initialize/applications/ExtendedForecast.py

Line 36 in ebe511b

'outIntervalHR': [12, int],

and replacing range in

MPAS-Workflow/initialize/applications/ExtendedForecast.py

Line 95 in ebe511b

self.ensVerifyMembers = range(1, self.NN+1, 1)

by a function that can handle floating point arguments (numpy.arange or similar). It is not a very elegant workaround and I suspect more things would break down the line.

Even if this should work, I think you raise a good point that our current workflow is built with the assumption of k * 1h increments (k is some integer). If we want to generalize to sub-hour, we could potentially relax the integer requirement and handle floating-point values themselves, or convert to HHMM format and handle the resulting integers.