Index Out Of Range during correct_troposphere step

[mayala02]

Description of the problem

I am trying to run the smallbaselineApp.py and I can successfully get into the velocity step and it crashes on this index out of bounds exception. Sometimes this fails with a division by zero error from the progress bar, but that is flaky and only happens sometimes. I'm not sure what the issue is, maybe it's an issue with the date format and trying to split on the "-" character.

Full script that generated the error
Running this command with these set in my config with the latest version of MintPy

(coastal-py) melanieayala@Melanies-Air CoastalEnrichment % cat mintpyConfigFile.txt
mintpy.load.processor        = hyp3
##---------interferogram datasets:
mintpy.load.unwFile          = sabineBU_rawData/*/*unw_phase_crop.tif
mintpy.load.corFile          = sabineBU_rawData/*/*corr_crop.tif
##---------geometry datasets:
mintpy.load.demFile          = sabineBU_rawData/*/*dem_crop.tif
mintpy.load.incAngleFile     = sabineBU_rawData/*/*lv_theta_crop.tif
mintpy.load.azAngleFile      = sabineBU_rawData/*/*lv_phi_crop.tif
mintpy.load.waterMaskFile    = sabineBU_rawData/*/*water_mask_crop.tif

mintpy.troposphericDelay.method = pyaps
mintpy.troposphericDelay.weatherModel = ERA5
(coastal-py) melanieayala@Melanies-Air CoastalEnrichment % smallbaselineApp.py mintpyConfigFile.txt

Full error message

******************** step - correct_troposphere ********************
Input data seems to be geocoded. Lookup file not needed.
Atmospheric correction using Weather Re-analysis dataset (PyAPS, Jolivet et al., 2011)
Weather Re-analysis dataset: ERA5
--------------------------------------------
Use existed tropospheric delay file: ./inputs/ERA5.h5

diff.py /Users/melanieayala/Documents/CoastalEnrichment/timeseries.h5 ./inputs/ERA5.h5 -o /Users/melanieayala/Documents/CoastalEnrichment/timeseries_ERA5.h5 --force
/Users/melanieayala/Documents/CoastalEnrichment/timeseries.h5 - ['./inputs/ERA5.h5'] --> /Users/melanieayala/Documents/CoastalEnrichment/timeseries_ERA5.h5
the 1st input file is: timeseries
WARNING: ./inputs/ERA5.h5 does not contain all dates in /Users/melanieayala/Documents/CoastalEnrichment/timeseries.h5
Continue and enforce the differencing for their shared dates only.
	with following dates are ignored for differencing:
['20191001', '20181018', '20181123', '20191106', '20200901', '20190814', '20200317', '20181111', '20200820', '20190510', '20190826', '20190416', '20200117', '20200516', '20200410', '20190404', '20200715', '20180831', '20200329', '20191212', '20190203', '20191013', '20190802', '20190615', '20180912', '20181217', '20191130', '20181229', '20181030', '20200504', '20200703', '20200925', '20200528', '20200808', '20200913', '20200727', '20190122', '20200609', '20200422', '20200129', '20200621', '20190627', '20190721', '20180924', '20190522', '20200210', '20181006', '20191118', '20190919', '20190709', '20190907', '20190110']
Traceback (most recent call last):
  File "/opt/homebrew/anaconda3/envs/coastal-py/bin/smallbaselineApp.py", line 10, in <module>
    sys.exit(main())
  File "/opt/homebrew/anaconda3/envs/coastal-py/lib/python3.10/site-packages/mintpy/cli/smallbaselineApp.py", line 208, in main
    run_smallbaselineApp(inps)
  File "/opt/homebrew/anaconda3/envs/coastal-py/lib/python3.10/site-packages/mintpy/smallbaselineApp.py", line 1117, in run_smallbaselineApp
    app.run(steps=inps.runSteps)
  File "/opt/homebrew/anaconda3/envs/coastal-py/lib/python3.10/site-packages/mintpy/smallbaselineApp.py", line 898, in run
    self.run_tropospheric_delay_correction(sname)
  File "/opt/homebrew/anaconda3/envs/coastal-py/lib/python3.10/site-packages/mintpy/smallbaselineApp.py", line 642, in run_tropospheric_delay_correction
    mintpy.cli.diff.main(iargs)
  File "/opt/homebrew/anaconda3/envs/coastal-py/lib/python3.10/site-packages/mintpy/cli/diff.py", line 86, in main
    diff_file(
  File "/opt/homebrew/anaconda3/envs/coastal-py/lib/python3.10/site-packages/mintpy/diff.py", line 390, in diff_file
    diff_timeseries(file1, file2[0], out_file, force_diff, max_num_pixel)
  File "/opt/homebrew/anaconda3/envs/coastal-py/lib/python3.10/site-packages/mintpy/diff.py", line 100, in diff_timeseries
    ref_val = readfile.read(file2, datasetName=date_list_shared, box=ref_box)[0] * unit_fac
  File "/opt/homebrew/anaconda3/envs/coastal-py/lib/python3.10/site-packages/mintpy/utils/readfile.py", line 287, in read
    dsname4atr = datasetName[0].split('-')[0]
IndexError: list index out of range

System information

• Operating system: macOS M2
• Python environment: conda
• MintPy version: MintPy version .1.20, date 2023-11-11
• InSAR processor/product: hyp3
• Your custom / default template file (if the bug is related to a specific dataset):
  mintpy.load.processor = hyp3
  ##---------interferogram datasets:
  mintpy.load.unwFile = sabineBU_rawData/*/unw_phase_crop.tif
  mintpy.load.corFile = sabineBU_rawData//corr_crop.tif
  ##---------geometry datasets:
  mintpy.load.demFile = sabineBU_rawData//dem_crop.tif
  mintpy.load.incAngleFile = sabineBU_rawData//lv_theta_crop.tif
  mintpy.load.azAngleFile = sabineBU_rawData//lv_phi_crop.tif
  mintpy.load.waterMaskFile = sabineBU_rawData//*water_mask_crop.tif

mintpy.troposphericDelay.method = pyaps
mintpy.troposphericDelay.weatherModel = ERA5

[welcome[bot]]

👋 Thanks for opening your first issue here! Please filled out the template with as much details as possible. We appreciate that you took the time to contribute!
Make sure you read our contributing guidelines.

[yunjunz]

The message says ERA5 datasets are not downloaded for many dates. Have you setup your ERA5 account as written out in insarlab/PyAPS repo? It's good to test the account for data downloading to ensure as well.