Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update harmony-service lib and update docker platform to amd64 #33

Merged
merged 11 commits into from
Oct 16, 2024
8 changes: 4 additions & 4 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,20 @@ ci:
autofix_prs: false
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.6.0
rev: v5.0.0
flamingbear marked this conversation as resolved.
Show resolved Hide resolved
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-json
- id: check-yaml
- id: check-added-large-files
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.5.5
rev: v0.6.9
hooks:
- id: ruff
args: ["--fix", "--show-fixes", "--select", "I"]
args: ["--fix", "--show-fixes", "--extend-select", "I"]
- repo: https://github.com/psf/black-pre-commit-mirror
rev: 24.4.2
rev: 24.10.0
hooks:
- id: black-jupyter
args: ["--skip-string-normalization"]
Expand Down
11 changes: 10 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,14 @@ HyBIG follows semantic versioning. All notable changes to this project will be
documented in this file. The format is based on [Keep a
Changelog](http://keepachangelog.com/en/1.0.0/).

## [v2.0.1] - 2024-10-06

### Changed

* Updates service image to be built on AMD64.
* Updates internal libraries


## [v2.0.0] - 2024-07-19

**DAS-2180** - Adds pip installable library.
Expand Down Expand Up @@ -62,7 +70,8 @@ outlined by the NASA open-source guidelines.
For more information on internal releases prior to NASA open-source approval,
see legacy-CHANGELOG.md.

[unreleased]:https://github.com/nasa/harmony-browse-image-generator/compare/2.0.0..HEAD
[unreleased]:https://github.com/nasa/harmony-browse-image-generator/compare/2.0.1..HEAD
[v2.0.1]:https://github.com/nasa/harmony-browse-image-generator/compare/2.0.0..2.0.1
[v2.0.0]:https://github.com/nasa/harmony-browse-image-generator/compare/1.2.2..2.0.0
[v1.2.2]: https://github.com/nasa/harmony-browse-image-generator/compare/1.2.1..1.2.2
[v1.2.1]: https://github.com/nasa/harmony-browse-image-generator/compare/1.2.0..1.2.1
Expand Down
3 changes: 2 additions & 1 deletion bin/build-image
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
# 2022-01-03: Updated Dockerfile path and added comments for tags.
# 2023-04-04: Updated for the Harmony Browse Image Generator (HyBIG).
# 2024-01-22: Updated image name to: ghcr.io/nasa/harmony-browse-image-generator.
# 2024-10-06: Updated to support amd64.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are these actually valuable or just something we keep doing?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good question. I was wondering something similar when I was updating the HGA build-image script. Basically, we're just recreating the commit history in the shell script, so probably they are not uniquely valuable.

Perhaps the one minor value is where the script was originally copied from. But that said, we're trying to standardise, so that shouldn't really matter.

I'm entirely ambivalent beyond saying if we have these comments, they should be complete, or we should not have them. It would be a bit frustrating to have a log of changes to the file and then subsequent changes not captured in them.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree. I'll pull them next time I visit (if I remember)

#
###############################################################################

Expand All @@ -29,4 +30,4 @@ tag=${1:-latest}
# version number from `docker/service_version.txt`.
# - "latest", so the test Dockerfile can use the service image as a base image.
#
docker build -t ${image}:${tag} -t ${image}:latest -f docker/service.Dockerfile .
docker build --platform linux/amd64 -t ${image}:${tag} -t ${image}:latest -f docker/service.Dockerfile .
3 changes: 2 additions & 1 deletion bin/build-test
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
# 2022-01-03: Updated Dockerfile path.
# 2023-04-04: Updated for the Harmony Browse Image Generator (HyBIG).
# 2024-01-22: Updated image name to: ghcr.io/nasa/harony-browse-image-generator-test.
# 2024-10-06: Updated to support amd64.
#
###############################################################################

Expand All @@ -26,4 +27,4 @@ if [ ! -z "$old" ] && [ "$2" != "--no-delete" ]; then
fi

# Build the image
docker build -t ${image}:${tag} -f docker/tests.Dockerfile .
docker build --platform linux/amd64 -t ${image}:${tag} -f docker/tests.Dockerfile .
3 changes: 2 additions & 1 deletion bin/run-test
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
# 2021-01-05: Updated for JUnit output.
# 2023-04-04: Updated for the Harmony Browse Image Generator (HyBIG).
# 2024-01-23: Updated to use new GitHub-related images.
# 2024-10-06: Updated to support amd64.
#
###############################################################################

Expand All @@ -24,7 +25,7 @@ mkdir -p coverage

# Run the tests in a Docker container with mounted volumes for XML report
# output and test coverage reporting
docker run --rm \
docker run --platform linux/amd64 --rm \
-v $(pwd)/test-reports:/home/tests/reports \
-v $(pwd)/coverage:/home/tests/coverage \
ghcr.io/nasa/harmony-browse-image-generator-test "$@"
2 changes: 1 addition & 1 deletion docker/service_version.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
2.0.0
2.0.1
4 changes: 2 additions & 2 deletions docs/HyBIG-Example-Usage.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -57,11 +57,11 @@
"metadata": {},
"outputs": [],
"source": [
"from harmony import Collection, Environment, Client, Request\n",
"\n",
"# creates an output directory for the downloaded files\n",
"from pathlib import Path\n",
"\n",
"from harmony import Client, Collection, Environment, Request\n",
"\n",
"output_dir = Path('./hybig-output')\n",
"Path.mkdir(output_dir, exist_ok=True)"
]
Expand Down
2 changes: 1 addition & 1 deletion docs/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
notebook==7.0.3
notebook==7.2.2
harmony-py~=0.4.14
2 changes: 1 addition & 1 deletion harmony_service/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
from argparse import ArgumentParser
from sys import argv

from harmony import is_harmony_cli, run_cli, setup_cli
from harmony_service_lib import is_harmony_cli, run_cli, setup_cli

from .adapter import BrowseImageGeneratorAdapter
from .exceptions import SERVICE_NAME
Expand Down
13 changes: 9 additions & 4 deletions harmony_service/adapter.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,20 @@
from shutil import rmtree
from tempfile import mkdtemp

from harmony import BaseHarmonyAdapter
from harmony.message import Source as HarmonySource
from harmony.message_utility import (
from harmony_service_lib import BaseHarmonyAdapter
from harmony_service_lib.message import Source as HarmonySource
from harmony_service_lib.message_utility import (
Comment on lines +15 to +17
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes me happy! I'm so grateful Danny pushed this change through.

has_crs,
has_scale_extents,
has_scale_sizes,
has_valid_scale_extents,
)
from harmony.util import bbox_to_geometry, download, generate_output_filename, stage
from harmony_service_lib.util import (
bbox_to_geometry,
download,
generate_output_filename,
stage,
)
from pystac import Asset, Catalog, Item

from harmony_service.exceptions import HyBIGInvalidMessageError, HyBIGServiceError
Expand Down
2 changes: 1 addition & 1 deletion harmony_service/exceptions.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
"""Module defining harmony service errors raised by HyBIG service."""

from harmony.util import HarmonyException
from harmony_service_lib.util import HarmonyException

SERVICE_NAME = 'harmony-browse-image-generator'

Expand Down
4 changes: 2 additions & 2 deletions hybig/browse.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@
import numpy as np
import rasterio
from affine import dumpsw
from harmony.message import Message as HarmonyMessage
from harmony.message import Source as HarmonySource
from harmony_service_lib.message import Message as HarmonyMessage
from harmony_service_lib.message import Source as HarmonySource
from matplotlib.cm import ScalarMappable
from matplotlib.colors import Normalize
from numpy import ndarray
Expand Down
2 changes: 1 addition & 1 deletion hybig/browse_utility.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
"""Module containing utility functionality for browse generation."""

from harmony.message import Message as HarmonyMessage
from harmony_service_lib.message import Message as HarmonyMessage


def get_harmony_message_from_params(params: dict | None) -> HarmonyMessage:
Expand Down
2 changes: 1 addition & 1 deletion hybig/color_utility.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

import numpy as np
import requests
from harmony.message import Source as HarmonySource
from harmony_service_lib.message import Source as HarmonySource
from osgeo_utils.auxiliary.color_palette import ColorPalette
from pystac import Item
from rasterio.io import DatasetReader
Expand Down
2 changes: 1 addition & 1 deletion hybig/crs.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

"""

from harmony.message import SRS
from harmony_service_lib.message import SRS
from pyproj.crs import CRS as pyCRS

# pylint: disable-next=no-name-in-module
Expand Down
8 changes: 6 additions & 2 deletions hybig/sizes.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,12 @@

import numpy as np
from affine import Affine
from harmony.message import Message
from harmony.message_utility import has_dimensions, has_scale_extents, has_scale_sizes
from harmony_service_lib.message import Message
from harmony_service_lib.message_utility import (
has_dimensions,
has_scale_extents,
has_scale_sizes,
)

# pylint: disable-next=no-name-in-module
from rasterio.crs import CRS
Expand Down
8 changes: 4 additions & 4 deletions pip_requirements.txt
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
harmony-service-lib~=1.0.27
harmony-service-lib~=2.0.0
matplotlib==3.9.0
numpy==1.26.4
pillow==10.3.0
pillow==10.4.0
pyproj==3.6.1
pystac~=0.5.6
pystac~=1.0.1
rasterio==1.3.10
rioxarray==0.15.5
rioxarray==0.17.0
4 changes: 2 additions & 2 deletions tests/test_service/test_adapter.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@
from unittest.mock import call, patch

import numpy as np
from harmony.message import Message
from harmony.util import config
from harmony_service_lib.message import Message
from harmony_service_lib.util import config
from pystac import Catalog
from rasterio.transform import array_bounds, from_bounds
from rasterio.warp import Resampling
Expand Down
4 changes: 2 additions & 2 deletions tests/test_service/unit/test_adapter_unit.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
from unittest import TestCase
from unittest.mock import Mock

from harmony.message import Message
from harmony.util import config
from harmony_service_lib.message import Message
from harmony_service_lib.util import config
from pystac import Asset, Item

from harmony_service.adapter import BrowseImageGeneratorAdapter
Expand Down
5 changes: 2 additions & 3 deletions tests/unit/test_browse.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,8 @@
from unittest.mock import MagicMock, Mock, call, patch

import numpy as np
from harmony.message import SRS
from harmony.message import Message as HarmonyMessage
from harmony.message import Source as HarmonySource
from harmony_service_lib.message import Message as HarmonyMessage
from harmony_service_lib.message import Source as HarmonySource
from numpy.testing import assert_array_equal
from osgeo_utils.auxiliary.color_palette import ColorPalette
from PIL import Image
Expand Down
2 changes: 1 addition & 1 deletion tests/unit/test_color_utility.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
from unittest import TestCase
from unittest.mock import Mock, patch

from harmony.message import Source as HarmonySource
from harmony_service_lib.message import Source as HarmonySource
from osgeo_utils.auxiliary.color_palette import ColorPalette
from pystac import Asset, Item
from rasterio import DatasetReader
Expand Down
2 changes: 1 addition & 1 deletion tests/unit/test_crs.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
from unittest.mock import patch

from affine import Affine
from harmony.message import SRS
from harmony_service_lib.message import SRS
from rasterio.crs import CRS
from rioxarray import open_rasterio

Expand Down
2 changes: 1 addition & 1 deletion tests/unit/test_sizes.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
from unittest.mock import MagicMock, patch

import rasterio
from harmony.message import Message
from harmony_service_lib.message import Message
from rasterio import Affine
from rasterio.crs import CRS
from rioxarray import open_rasterio
Expand Down
2 changes: 1 addition & 1 deletion tests/utilities.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
from collections import namedtuple
from datetime import datetime

from harmony.util import bbox_to_geometry
from harmony_service_lib.util import bbox_to_geometry
from pystac import Asset, Catalog, Item

Granule = namedtuple('Granule', ['url', 'media_type', 'roles'])
Expand Down