Replace the functionality of ncap2 in the quality.sh script

[jasonfleming]

The use of ncap2 from NCO is prohibitive due to its dependencies. The functionality it provides will need to be replaced.

[wwlwpd]

Can you summarize what the following is impliclty doing via the ncap2 one liners - is this kind of like awk but for netCDF files?

From:

https://github.com/StormSurgeLive/asgs/blob/master/util/output/quality.sh

    # compute min, max, avg, stdev, and number of missing (i.e., dry) values
    # over the full domain for each time step and write to file
    for snap in $(seq 0 $((${filesNumDataSets[$file]} - 1))) ; do
        if [[ $file == "maxele.63.nc" || $file == "maxwvel.63.nc" || $file == "swan_HS_max.63.nc" ]]; then
            ncap2 -O -v -s "data[node]=0.0;data.set_miss(-99999.0);data=${netcdfVarName[$file]}(0:$(($np - 1)):1);print(data.min(),\"%f \");print(data.max(),\"%f \");print(data.avg(),\"%f \");print((data-data.avg()).rmssdn(),\"%f \");print(data.number_miss(),\"%d\n\");" $file statistics.nc >> statistics_${file}.txt
        else
            ncap2 -O -v -s "data[node]=0.0;data.set_miss(-99999.0);data=${netcdfVarName[$file]}($snap:$snap:1,0:$(($np - 1)):1);print(data.min(),\"%f \");print(data.max(),\"%f \");print(data.avg(),\"%f \");print((data-data.avg()).rmssdn(),\"%f \");print(data.number_miss(),\"%d\n\");" $file statistics.nc >> statistics_${file}.txt
        fi
    done

    "maxele.63.nc"|"fort.63.nc")
        # count the number of nodes that would be considered dry upon cold start based only on negative topobathy value
        numLand=$(ncap2 -O -v -s "land_msk[node]=0.0;land_msk.set_miss(-99999.0);where(depth < 0.0) land_msk=-99999.0; elsewhere land_msk=depth; print(land_msk.number_miss(),\"%d\n\");" $file statistics.nc)

[wwlwpd]

FWIW, antlr2 is a parser generator, so maybe it's possible to run the build command to some early intermediate step or maybe the code author can generate the intermediate files and not require anyone building it to do so? I'll email him.

[wwlwpd]

They want to be contacted through the forums, so here's my request - https://sourceforge.net/p/nco/discussion/9829/thread/2ab575e3ef/

[jasonfleming]

ncap2 has its own mini-scripting language which is a little weird but it has some neat features. Those lines that invoke ncap2 in your comment above are looping over the datasets in the netCDF, counting the number of missing values in each one, and also computing the max, min, avg, and standard deviation.

I took a look at antlr2 after your comment on the previous issue and honestly antlr2 looks like a nightmare that is not remotely worth the trouble. I had no idea that nco even had external dependencies (other than netCDF itself) until now.

I think it will be straightforward to reimplement the little ncap2 script from quality.sh using the other netCDF helper libraries we already have in ASGS. I'm hoping to knock that out later this week. :-)