Writing adjusted variables and flags

[cgrdn]

Functionality to read in, manipulate, and calculate gain of oxygen has been built, but the end goal will be to adjust oxygen and/or the associated quality flags. Keep in mind that populating the _ADJUSTED field requires a change in DATA_MODE along with the population of _ADJUSTED_QC and _ADJUSTED_ERROR.

Plan to build the functions to write adjusted quantities and change or create new quality flags, but will not write to actual float file for now, will instead write to a local-only file with a Q*.nc prefix so that they are easy to keep track of, but this can be easily replaced with overwriting the file when the library is available for real use.

[cgrdn]

This issue (along with #12) are operational priorities following today's release.

[cgrdn]

Note for changing SCIENTIFIC_CALIB type parameters, N_CALIB is the dimension that will need to be adjusted. This will take some file I/O hoop jumping through for the following variables:

for v in nc.variables.keys():
    if 'N_CALIB' in nc.variables[v].dimensions:
        print(v)
 
Out: 

PARAMETER
SCIENTIFIC_CALIB_EQUATION
SCIENTIFIC_CALIB_COEFFICIENT
SCIENTIFIC_CALIB_COMMENT
SCIENTIFIC_CALIB_DATE

[cgrdn]

netCDF files are, by design, difficult to alter. For the above variables I'll be following the example in this post to update them: https://stackoverflow.com/questions/15141563/python-netcdf-making-a-copy-of-all-variables-and-attributes-but-one

I will also have to exclude the N_CALIB dimension to iterate it.

[cgrdn]

Implemented the above in 007ba5a. The basic work flow will look like:

• take a netCDF file you want to alter/add to
• create a copy of everything except what you want to change
• add dimensions/variables that are different, likely appending to old ones
• save re-written file

[cgrdn]

Just did some testing and first step appears to be working well. Worth noting that if you are going to exclude a dimension, you must also exclude any variables that have that dimension. This could be done automatically quite easily, but I am going to leave it without doing that so that the user is required to be specific in what they are doing. Else, you could delete a variable you don't realize uses that dimension. I will however add in some good/informative output for situations when that does occur (i.e. strong error handling if a variable uses a dimension that no longer exists).

[cgrdn]

Function to append new variable was also written in f35d817. In practice, will likely delete a variable and dimension using the above function, iterate the dimension (likely N_CALIB += 1) and add to an existing nc variable. Once testing is done on this function, will consider this issue closed. The next step will be to check the altered files against the Argo file checker, will create separate issue for that.

[cgrdn]

For reference, as adding to SCIENTIFIC_CALIB_<> will be the main purpose of this function, standard comments via Henry Bittig at ADMT21:

PARAM	Common Notation
SCIENTIFIC_CALIB_EQUATION	PPOX_DOXY=f(DOXY); PPOX_DOXY_ADJUSTED = OFFSET + (SLOPE*(1+DRIFT/100*(profile_date_juld–launch_date_juld)/365) + INCLINE_T*TEMP)*PPOX_DOXY; DOXY_ADJUSTED=f(PPOX_DOXY_ADJUSTED)
SCIENTIFIC_CALIB_COEFFICIENT	OFFSET=a; SLOPE=b; DRIFT=c; INCLINE_T=d; launch_date_juld=yyyymmddHHMMSS
SCIENTIFIC_CALIB_COMMENT	PPOX converted from DOXY; PPOX corrected [PART2]; DOXY_ADJUSTED converted from PPOX_DOXY_ADJUSTED; DOXY_ADJUSTED_ERROR recomputed from a PPOX_DOXY_ADJUSTED_ERROR = [xx] mbar

Issue	Prevelance	Remedy
Bias in O2 sensitivity	All Floats with Optodes	SLOPE Factor on PPOX_DOXY
Ongoing change in O2 sensitivity (“in situ drift”)	Many Floats (>50 %)	Time-dependent DRIFT in Slope Factor on PPOX_DOXY
Improper/Incomplete Temperature Compensation	All 2nd Generation Floats (a)	Temperature-dependent Slope Factor by INCLINE_T
Improper/Incomplete Oxygen Compensation at low O2 values	All 2nd Generation Floats (a)	(Small) OFFSET on PPOX_DOXY
Improper/Incomplete Pressure Compensation	unknown (mostly Deep Argo-O2?)	Refinement of Pcoef3; Addition of INCLINE_P?

(a): 1st Generation: Electrochemical SBE43; 2nd Generation: Batch-Foil calibrated Optodes; 3rd Generation: Multi-Point calibrated Optodes

[cgrdn]

Testing in 3e6f987 passes the Argo FileChecker. Still need to add the functionality in that script into the package, but this is a big milestone. Will keep this issue open until standard comments and functions are built, but am going to consider the task in #41 related to this issue complete.

[cgrdn]

The name of the issue no longer is representative, since following the implementation of iterate_dimension (#57), this is a one-line action in a script. Still though, mission accomplished, so closing.