Skip to content

Commit

Permalink
Merge branch 'develop' into add-dependabot-file
Browse files Browse the repository at this point in the history
  • Loading branch information
mfixstsci authored Oct 23, 2023
2 parents 2f48901 + 46d2548 commit f73a012
Show file tree
Hide file tree
Showing 124 changed files with 9,795 additions and 5,410 deletions.
87 changes: 31 additions & 56 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,78 +2,53 @@ name: JWQL CI

on: [push, pull_request]

jobs:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

JWQL-CI:
defaults:
run:
shell: bash -l {0}

name: Python - ${{ matrix.python-version }}
jobs:
check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: "3.x"
- run: pip install bandit
- run: bandit ./jwql/ -c .bandit

test:
name: test (Python ${{ matrix.python-version }}, ${{ matrix.os }})
runs-on: ${{ matrix.os }}
strategy:
max-parallel: 5
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest]
python-version: [3.8, 3.9]
python-version: [3.9, "3.10"]

steps:
- uses: actions/checkout@v3

- uses: conda-incubator/setup-miniconda@v2
- uses: mamba-org/provision-with-micromamba@v15
with:
activate-environment: jwql-${{ matrix.python-version }}
environment-file: ./environment_python_${{ matrix.python-version }}.yml
cache-env: true
cache-downloads: true

- name: Bandit Check
uses: jpetrucciani/bandit-check@master
with:
path: "./jwql/"
bandit_flags: "-c .bandit"
continue-on-error: false
if: runner.os == 'Linux'
- run: pip install -e .[test]

- name: Start Redis
uses: supercharge/[email protected]
- run: conda env export

- uses: supercharge/[email protected]
with:
redis-version: 5.0
if: runner.os == 'Linux'

- name: Build jwql conda environment and run tests
shell: bash -l {0}
run: |
echo "Setting useful environment variables:"
echo " "
export PYTHONVERSION=${{ matrix.python-version }}
export MAJOR="$(echo $PYTHONVERSION | head -c 1)"
export MINOR="$(echo $PYTHONVERSION | tail -c 2)"
echo "PYTHONVERSION: $PYTHONVERSION"
echo " "
echo "Installing jwql conda environment"
echo " "
cd $RUNNER_WORKSPACE/
cd jwql/
$CONDA/bin/conda env update -f environment_python_${MAJOR}_${MINOR}.yml -n jwql-${{ matrix.python-version }}
echo " "
echo "Installing jwql package"
echo " "
python setup.py develop
echo " "
echo "Testing package installation"
echo " "
python -c "import jwql; print('Version: ' + jwql.__version__); print('Path: ' + jwql.__path__[0])"
# echo " "
# echo "Reinstall numpy to fix numpy.core.multiarray error"
# echo " "
pip install -U numpy
echo " "
echo "The conda environment being used:"
echo " "
$CONDA/bin/conda env export
- run: python -c "import jwql; print('Version ' + jwql.__version__); print('Path ' + jwql.__path__[0])"

echo " "
echo "Running pytests"
echo " "
pytest jwql/tests/
- run: pytest jwql/tests/
4 changes: 2 additions & 2 deletions .readthedocs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ build:
python: "3.9"
jobs:
post_install:
- pip install sqlalchemy==1.4.43
- pip install sqlalchemy==1.4.46

# Build documentation in the docs/ directory with Sphinx
sphinx:
Expand All @@ -29,4 +29,4 @@ python:
install:
- requirements: rtd_requirements.txt
- method: pip
path: .
path: .
2 changes: 1 addition & 1 deletion MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ include LICENSE
include CHANGES.rst
include environment_python_3_7.yml
include environment_python_3_8.yml
include setup.py
include pyproject.toml
include requirements.txt
include rtd_requirements.txt
include jwql/example_config.json
Expand Down
15 changes: 8 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,12 +44,12 @@ Getting `jwql` up and running on your own computer requires four steps, detailed

### Prerequisites

It is highly suggested that contributors have a working installation of `anaconda` or `miniconda` for Python 3.8. Downloads and installation instructions are available here:
It is highly suggested that contributors have a working installation of `anaconda` or `miniconda` for Python 3.9+. Downloads and installation instructions are available here:

- [Miniconda](https://conda.io/miniconda.html)
- [Anaconda](https://www.continuum.io/downloads)

Requirements for contributing to the `jwql` package will be included in the `jwql` `conda` environment, which is included in our installation instructions below. Further package requirements will be provided for `jwql` by a `setup.py` script included in the repository.
Requirements for contributing to the `jwql` package will be included in the `jwql` `conda` environment, which is included in our installation instructions below. Further package requirements will be provided for `jwql` by a `pyproject.toml` file included in the repository.

### Clone the `jwql` repo

Expand Down Expand Up @@ -83,16 +83,16 @@ source activate base/root

**Note:** If you have added a step activating conda to your default terminal/shell (e.g. the `.bashrc`, `.zshrc`, or `.profile` file) then you don't need to do the above step.

Lastly, create the `jwql` environment via one of the `environment.yml` files (currently `environment_python_3_8.yml`, for python 3.8, and `environment_python_3.9.yml`, for python 3.9, are supported by `jwql`):
Lastly, create the `jwql` environment via one of the `environment.yml` files (currently `environment_python_3.9.yml`, for python 3.9, and `environment_python_3.10.yml`, for python 3.10, are supported by `jwql`):

```
conda env create -f environment_python_3_8.yml
conda env create -f environment_python_3.9.yml
```

or

```
conda env create -f environment_python_3_9.yml
conda env create -f environment_python_3.10.yml
```

### Configuration File
Expand Down Expand Up @@ -154,10 +154,11 @@ Any questions about the `jwql` project or its software can be directed to `jwql@
- Mees Fix (Technical Lead, INS) [@mfixstsci](https://github.com/mfixstsci)
- Misty Cracraft (INS) [@cracraft](https://github.com/cracraft)
- Mike Engesser (INS) [@mengesser](https://github.com/mengesser)
- Shannon Osborne (INS) [@shanosborne](https://github.com/shanosborne)
- Maria Pena-Guerrero [@penaguerrero](https://github.com/penaguerrero)
- Ben Sunnquist (INS) [@bsunnquist](https://github.com/bsunnquist)
- Brian York (INS) [@york-stsci](https://github.com/york-stsci)
- Bradley Sappington (INS) [@bradleysappington](https://github.com/bradleysappington)
- Melanie Clarke (INS) [@melanieclarke](https://github.com/melanieclarke)

## Past Development Team Members
- Matthew Bourque (INS) [@bourque](https://github.com/bourque)
Expand All @@ -168,7 +169,7 @@ Any questions about the `jwql` project or its software can be directed to `jwql@
- Sara Ogaz (DMD) [@SaOgaz](https://github.com/SaOgaz)
- Catherine Martlin (INS) [@catherine-martlin](https://github.com/catherine-martlin)
- Johannes Sahlmann (INS) [@Johannes-Sahlmann](https://github.com/johannes-sahlmann)

- Shannon Osborne (INS) [@shanosborne](https://github.com/shanosborne)

## Acknowledgments:
- Faith Abney (DMD)
Expand Down
6 changes: 0 additions & 6 deletions docs/source/jwql_monitors.rst
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,3 @@ monitor_filesystem.py
.. automodule:: jwql.jwql_monitors.monitor_filesystem
:members:
:undoc-members:

monitor_mast.py
---------------
.. automodule:: jwql.jwql_monitors.monitor_mast
:members:
:undoc-members:
4 changes: 2 additions & 2 deletions docs/source/tests.rst
Original file line number Diff line number Diff line change
Expand Up @@ -74,9 +74,9 @@ test_logging_functions.py
:members:
:undoc-members:

test_monitor_mast.py
test_mast_utils.py
--------------------
.. automodule:: jwql.tests.test_monitor_mast
.. automodule:: jwql.tests.test_mast_utils
:members:
:undoc-members:

Expand Down
8 changes: 1 addition & 7 deletions docs/source/utils.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,6 @@
utils
*****

anomaly_query_config.py
-----------------------
.. automodule:: jwql.utils.anomaly_query_config
:members:
:undoc-members:

calculations.py
---------------
.. automodule:: jwql.utils.calculations
Expand Down Expand Up @@ -90,4 +84,4 @@ utils.py
--------
.. automodule:: jwql.utils.utils
:members:
:undoc-members:
:undoc-members:
6 changes: 0 additions & 6 deletions docs/source/website.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,6 @@ data_containers.py
:members:
:undoc-members:

db.py
-----
.. automodule:: jwql.website.apps.jwql.db
:members:
:undoc-members:

forms.py
--------
.. automodule:: jwql.website.apps.jwql.forms
Expand Down
73 changes: 73 additions & 0 deletions environment_python_3.10.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# This file describes a conda environment that can be to install jwql
#
# Run the following command to set up this environment:
# $ conda env create -f environment_python_3.10.yml
#
# The environment name can be overridden with the following command:
# $ conda env create -n <custom name> -f environment_python_3.10.yml
#
# Run the following command to activate the environment:
# $ source activate jwql-3.10
#
# To deactivate the environment run the following command:
# $ source deactivate
#
# To remove the environment entirely, run the following command:
# $ conda env remove -n jwql-3.10

name: jwql-3.10

channels:
- conda-forge
- defaults

dependencies:
- astropy=5.3.4
- beautifulsoup4=4.12.2
- bokeh=2.4.3
- celery=5.3.4
- cryptography=41.0.4
- django=4.2.6
- inflection=0.5.1
- ipython=8.16.1
- jinja2=3.1.2
- jsonschema=4.19.1
- matplotlib=3.8.0
- nodejs=20.8.0
- numpy=1.25.2
- numpydoc=1.5.0
- pandas=2.1.1
- pip=23.2.1
- postgresql=15.4
- psycopg2=2.9.7
- pytest=7.4.2
- pytest-cov=4.1.0
- pytest-mock=3.11.1
- python=3.10.12
- pyyaml=6.0.1
- redis=5.0.0
- ruff=0.0.292
- scipy=1.9.3
- setuptools=68.2.2
- sphinx=7.2.6
- sphinx_rtd_theme=1.3.0
- sqlalchemy=2.0.21
- twine=4.0.2
- wtforms=3.0.1

- pip:
- astroquery==0.4.6
- bandit==1.7.5
- jwst==1.12.3
- pysiaf==0.20.0
- pysqlite3==0.5.2
- pyvo==1.4.2
- redis==5.0.0
- selenium==4.13.0
- stdatamodels==1.8.3
- stsci_rtd_theme==1.0.0
- vine==5.0.0
- git+https://github.com/spacetelescope/jwst_reffiles

# Current package
- -e .
73 changes: 73 additions & 0 deletions environment_python_3.9.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# This file describes a conda environment that can be to install jwql
#
# Run the following command to set up this environment:
# $ conda env create -f environment_python_3.9.yml
#
# The environment name can be overridden with the following command:
# $ conda env create -n <custom name> -f environment_python_3.9.yml
#
# Run the following command to activate the environment:
# $ source activate jwql-3.9
#
# To deactivate the environment run the following command:
# $ source deactivate
#
# To remove the environment entirely, run the following command:
# $ conda env remove -n jwql-3.9

name: jwql-3.9

channels:
- conda-forge
- defaults

dependencies:
- astropy=5.3.3
- beautifulsoup4=4.12.2
- bokeh=2.4.3
- celery=5.3.4
- cryptography=41.0.4
- django=4.2.5
- inflection=0.5.1
- ipython=8.16.1
- jinja2=3.1.2
- jsonschema=4.19.1
- matplotlib=3.8.0
- nodejs=20.8.0
- numpy=1.25.2
- numpydoc=1.5.0
- pandas=2.1.1
- pip=23.2.1
- postgresql=15.4
- psycopg2=2.9.7
- pytest=7.4.2
- pytest-cov=4.1.0
- pytest-mock=3.11.1
- python=3.9.17
- pyyaml=6.0.1
- redis=5.0.0
- ruff=0.0.292
- scipy=1.9.3
- setuptools=68.2.2
- sphinx=7.2.6
- sphinx_rtd_theme=1.3.0
- sqlalchemy=2.0.21
- twine=4.0.2
- wtforms=3.0.1

- pip:
- astroquery==0.4.6
- bandit==1.7.5
- jwst==1.12.3
- pysiaf==0.20.0
- pysqlite3==0.5.2
- pyvo==1.4.2
- redis==5.0.0
- selenium==4.13.0
- stdatamodels==1.8.3
- stsci_rtd_theme==1.0.0
- vine==5.0.0
- git+https://github.com/spacetelescope/jwst_reffiles

# Current package
- -e .
Loading

0 comments on commit f73a012

Please sign in to comment.