missing xml job dependency for gdas_marineanlletkf

[RussTreadon-NOAA]

What is wrong?

Job gdas_marineanlletkf in g-w CI case C48mx500_hybAOWCDA fails when gdas_marineanlletkf runs before gdas_marinebmat.

gdas_marineanlletkf uses file ../ensdata/ens/ocean.1.nc as input. This file is created by gdas_marinebmat. When gdas_marineanlletkf runs before gdas_marinebmat, the letkf job fails with the error message

 0:
 0: FATAL from PE     0: fms_io(restore_state_all): unable to find any restart files specified by ../ensdata/ens/ocean.1.nc
 0:
 6:
 6: FATAL from PE     6: fms_io(restore_state_all): unable to find any restart files specified by ../ensdata/ens/ocean.1.nc
 6:
12:
12: FATAL from PE    12: fms_io(restore_state_all): unable to find any restart files specified by ../ensdata/ens/ocean.1.nc
12:

What should have happened?

gdas_marineanlletkf should successfully run to completion. This, indeed, happens when gdas_marinebmat completes before gdas_marineanlletkf is submitted.

What machines are impacted?

All or N/A

What global-workflow hash are you using?

4e73a31

Steps to reproduce

1. clone RussTreadon-NOAA:feature/update_gdas at 4e73a31
2. build and install g-w
3. set up g-w CI for C48mx500_hybAOWCDA
4. cycle C48mx500_hybAOWCDA to 2021032500.
5. let rocotorun boot gdas_marinebmat and gdas_marineanlletkf. Use debug queue for gdas_marineanlletkf to ensure quick turnaround. Put gdas_marinebmat in batch for slower turnaround.
6. gdas_marineanlletkf will fail when it runs before gdas_marinebmat completes.

Additional information

No response

Do you have a proposed solution?

Currently, C48mx500_hybAOWCDA.xml only includes the following dependencies for gdas_marineanlletkf

        <dependency>
                <and>
                        <metataskdep metatask="enkfgdas_fcst" cycle_offset="-06:00:00"/>
                        <taskdep task="gdas_prepoceanobs"/>
                </and>
        </dependency>

The entry

                        <taskdep task="gdas_marinebmat"/>

should be added below the gdas_prepoceanobs` line.

[RussTreadon-NOAA]

Attention @guillaumevernieres and @AndrewEichmann-NOAA

Would you please

1. confirm that the above reported bug is truly a bug
2. confirm that the proposed solution is correct

[guillaumevernieres]

Attention @guillaumevernieres and @AndrewEichmann-NOAA

Would you please

1. confirm that the above reported bug is truly a bug
2. confirm that the proposed solution is correct

Oops.

1. Yes, it's a bug
2. Correct-ish, the LETKF task should wait for the ensemble members to be staged, which is done in the marinbmat job.

[AndrewEichmann-NOAA]

@RussTreadon-NOAA I can do the paperwork

[RussTreadon-NOAA]

Thank you @AndrewEichmann-NOAA for following up on this issue. Feel free to list me as a reviewer when submit the PR.