Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report error when variable has no units and neither standard_name nor long_name #55

Open
atmodatcode opened this issue Sep 3, 2021 · 4 comments
Assignees
Labels
enhancement New feature or request

Comments

@atmodatcode
Copy link
Member

Request for enhancement:
When performing run_checks.py on a file with no variable attributes, there is no error report that this is not ATMODAT Standard compliant.
The CF Checker writes out a warning, but no error.
It is important to report missing variable units and either long_name or standard_name as an error.

e.g.
(atmodat) angelika ~/Checker/demo_data: run_checks.py -s -f NOATTRIBUTES.nc -op checkoutput
Running Compliance Checker on the datasets from: ['NOATTRIBUTES.nc']
2021-09-03 07:40:27.459085 [INFO] :: PYESSV :: Loading vocabularies from /Users/angelika/Checker/atmodat_data_checker/AtMoDat_CVs/pyessv-archive:
2021-09-03 07:40:27.466281 [INFO] :: PYESSV :: ... loaded: atmodat
--- 4.3694 seconds for checking 1 files---
(atmodat) angelika ~/Checker/demo_data: more checkoutput/short_summary.txt
Short summary of checks:

Checking against: atmodat_standard:3.0, CF table version: 77
Version of the AtMoDat checker: 1.1.0
Checked at: 2021-09-03T09:40:31.270952

Number of checked files: 1
Total checks passed: 0/29
Mandatory checks passed: 0/2
Recommended checks passed: 0/18
Optional checks passed: 0/9
CF checker errors: 0

@atmodatcode atmodatcode added the enhancement New feature or request label Sep 3, 2021
@atmodatcode
Copy link
Member Author

atmodatcode commented Oct 25, 2021

First priority would be to check the metadata of the data variables.

@jkretz jkretz self-assigned this Oct 25, 2021
@jkretz
Copy link
Collaborator

jkretz commented Oct 28, 2021

I was thinking about this issue a little and it is relatively hard to confidently extract all data variables. It is easy to exclude the variables that define dimensions. But we still have variables that, for example, define the boundaries of coordinate variables.

For a really well-prepared netCDF file, it would be possible to exclude such special variables. This is probably not the case for a data provider, especially when first trying to prepare data for submission. In such a case, our checker would give a lot of errors, out of which some might be misleading as they go way beyond what would be required by the CF Conventions (e.g., some variables might not need units or long_name).

Overall, I think we really have to discuss if we impose the above-mentioned requirements.

@atmodatcode
Copy link
Member Author

I think we should at least add some summary in *CF_result.txt to shortly inform data producers and data curators on the degree of standardisation of the data variables.
For example:

  • No standard_name or long_name attribute specified for variables: time, ps
  • No units specified for variables: time, ps
    Only showing
    "CF checker errors: 0"
    when the data variables contain no metadata is misleading.

Thanks.

@jkretz
Copy link
Collaborator

jkretz commented Jan 24, 2022

The checker will output the number of warnings encountered during checking with #111

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants