Skip to content

Commit

Permalink
Merge branch 'develop' into release/0.8.0
Browse files Browse the repository at this point in the history
  • Loading branch information
jamesfwood committed Nov 9, 2023
2 parents 3cb316a + 26d8a61 commit eab2bfe
Show file tree
Hide file tree
Showing 3 changed files with 43 additions and 13 deletions.
25 changes: 20 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,23 @@
# Table of contents

- [Overview](#overview)
- [Install poetry](#install-poetry-python-dependency-manager)
- [Build](#build)
- [How to load and use tig module](#how-to-load-and-use-tig-module)
- [tig Input](#tig-input)
- [tig Output](#tig-output)
- [CLI Commands](#cli-commands)

## Overview
tig is built to be used within Cumulus ecosystem. It is depending on cumulus CMA ([Cumulus Documentation](https://nasa.github.io/cumulus)).
Please refer to the [Usage](#usage) section for inputs and outputs. TIG itself is a lambda function which runs on top of CMA as its lambda layer.

## Install poetry (python dependency manager)
Install poetry following the directions here: https://python-poetry.org/docs/#installation

- Be sure to install poetry in an isolated environment from the rest of your system.
- Be sure to use with a python version less than 3.12

## Build
* Jenkins pipeline template is applied to this project.
* development is based on poetry python environment.
Expand All @@ -26,13 +34,18 @@ Unit Test can be run using the command
poetry run pytest
```

## Cli
Use cli to test thumbnail image generation for a granule with configuration file and palettes
## CLI Commands
- [Generate Thumbnails](#generate-thumbnails)
- [Generate Config File](#generate-config-file)
- [Run Tig](#run-tig)

### Generate Thumbnails
Use cli helper script to generate thumbnails for each collection (NOTE: This automatically generates the config file and runs tig)
```
tig --input_file <granule> --output_dir <output_dir> --config_file <config_file> --palette_dir <palette_dir>
Go to dir `podaac/tig` and look at generate_thumbnails.sh script. Follow directions at top to setup.
```

### Generate Config File
Use cli to create a tig configuration for collections
```
generate_hitide_config --granule <granule_file> -dataset-id <collection short name> --include-image-variables <csv file image variables> --longitude <lon variable> --latitude <lat variable> --time <time variable> --footprint_strategy <footprint strategy>
Expand All @@ -46,9 +59,11 @@ latitude: latitude variable include the group if they're in a group defaults to
time: time variable include the group if they're in a group defaults to time
footprint_strategy: strategy to generate footprint will default to None options should be ["periodic", "linestring", "polar", "swot_linestring", "polarsides", "smap"]

Use cli helper script to generate thumbnails for each collection
### Run Tig
Use cli to test thumbnail image generation for a granule with configuration file and palettes

```
Go to dir `podaac/tig` and look at generate_thumbnails.sh script. Follow directions at top to setup.
tig --input_file <granule> --output_dir <output_dir> --config_file <config_file> --palette_dir <palette_dir>
```

## How to load and use tig module
Expand Down
29 changes: 22 additions & 7 deletions podaac/tig/generate_thumbnails.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,35 +3,50 @@
# Prerequisites
#
# 1. Clone forge-tig-configuration into the parent dir of this repo
# 2. Setup poetry and run poetry install
# 2. Install poetry using this: https://python-poetry.org/docs/#installation
# 3. In a shell, cd to the tig root dir
# 4. Make sure poetry is setup to use python less than 3.12.
# Run commands:
# poetry env list
# poetry env remove <env id> (<< this is the environment id returned from 'poetry env list', need to remove only if it's version > 3.10)
# poetry env use python3.10
# 5. Run command "poetry install"

# HOW-TO Use
#
# Setup Collection Short Name Directories
#
# 1. Create a directory in the tig root dir named "configs"
# 2. Create directories under "configs" dir with the collection short name
# 2. For each collection, create a directory under "configs" dir with the collection short name
# e.g. in repo dir: configs/ASCATA_ESDR_L2_WIND_STRESS_V1.1
# 3. Copy this script to each short name dir
# 4. Copy file "example_vars.csv" (in repo root dir) to a file called "vars.csv" in each short name dir
# 5. Update vars.csv file in each collection short name specifically for that collection (i.e. update var names and min/maxes)
# 5. Update vars.csv file in each collection short name dir specifically for that collection (i.e. update var names and min/maxes)
# 6. Download a "representative" .nc granule file for that collection short name and move it into each short name dir
# 7. Update this script in each short name dir specifically for its collection short name (e.g. update input parameters)
# 7. Update this script in each short name dir specifically for its collection short name
# e.g. Update the input variable names or paths: --latitude lat TO --latitude data_01/ku/latitude
# e.g. Remove time variable
# e.g. Change footprint-strategy parameter
# e.g. Change tig's palette-dir parameter

# Run Script
#
# 1. Change dir to the configs/<shortname> dir
# 2. Run command "./generate_thumbnails.sh" to automatically generate the config, run tig, and open the thumbnails
# 3. The collection <shortname>.cfg file will be saved in each short name dir
# 4. Verify .cfg file and all thumbnail images in each short name dir
# 4. Verify <shortname>.cfg file and all thumbnail images in each short name dir
# 5. Tweak var.csv or this script in short name dir if needed and re-run
#
# NOTE: In rare cases, the generate_hitide_config.py file may need tweaking to generate a correct config file

# Publish config files to forge-tig-configuration repo
#
# 1. Copy each short name .cfg file to the forge-tig-configuration repo under "config-files"
# 2. Commit the new cfg files in forge-tig-configuration to a new branch named "feature/<name>"
# 3. Open a PR in forge-tig-configuration for this new branch
# 2. Update the CHANGELOG.md in forge-tig-configuration listing your changes
# 3. Commit and push the new updates in forge-tig-configuration to a new branch named "feature/<name>"
# NOTE: The <name> can be anything you want to describe the collection you are adding
# 4. Open a PR in forge-tig-configuration for this new branch
# 5. In the PR, include reviewers James Wood and Simon Liu, and let us know it's ready in Slack

collection=$(basename "$(pwd)")

Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "podaac-tig"
version = "0.8.0-alpha.4"
version = "0.9.0-alpha.1"
description = "Tool for Image Generation (TIG)"
authors = ["podaac-tva <[email protected]>"]
license = "Apache-2.0"
Expand Down

0 comments on commit eab2bfe

Please sign in to comment.