Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release resolve conflicts #117

Merged
merged 161 commits into from
Nov 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
161 commits
Select commit Hold shift + click to select a range
b8a76cd
change bars
mpound Jun 19, 2023
ce911f8
Merge branch 'main' of github.com:GreenBankObservatory/dysh
mpound Jun 29, 2023
c95bb5e
Merge branch 'main' of github.com:GreenBankObservatory/dysh
mpound Jul 28, 2023
def27c3
Merge branch 'main' of github.com:GreenBankObservatory/dysh
mpound Jul 28, 2023
d64fda5
Fix syntax error in release.yaml
tchamberlin Aug 23, 2023
20b137d
Merge pull request #72 from GreenBankObservatory/main-fix-release-action
tchamberlin Aug 23, 2023
993ee64
Updates to pre-commit hooks and pyproject config
tchamberlin Aug 25, 2023
bf3c6a0
Run all pre-commit hooks
tchamberlin Aug 25, 2023
c5eeb83
Fix invalid escape sequences
tchamberlin Aug 25, 2023
5bc5295
Fix a few linting errors
tchamberlin Aug 25, 2023
69fd204
Add pre-commit GitHub action
tchamberlin Aug 25, 2023
356cb9e
Don't emit trusted host in pip-compile
tchamberlin Aug 25, 2023
a3995c1
Remove trusted host line from requirements.txt
tchamberlin Aug 25, 2023
565aa79
Disable ruff pre-commit hook
tchamberlin Aug 25, 2023
ec6904f
Remove requirements_dev.txt; not used
tchamberlin Aug 25, 2023
db222b1
Merge pull request #74 from GreenBankObservatory/main-run-precommit
mpound Aug 28, 2023
98601e7
Merge branch 'main' of github.com:GreenBankObservatory/dysh
mpound Sep 8, 2023
4f65c5c
Update python-package.yml
astrofle Sep 20, 2023
b4d75c5
change index to be over all bintables
mpound Sep 25, 2023
157d631
change default python to 3.10
mpound Sep 25, 2023
f5a4ee6
cleanup
mpound Sep 25, 2023
def8687
get working for multi-bintable index
mpound Sep 25, 2023
8391181
add Scan base class
mpound Sep 25, 2023
e30a64f
start work on multi-file gbtfitsload
mpound Sep 26, 2023
c9cc279
change Scan to ScanMixin
mpound Sep 28, 2023
f881c33
doc change index() function
mpound Sep 28, 2023
b1f7154
state save after precommit
mpound Oct 3, 2023
38cd9c8
Merge pull request #81 from GreenBankObservatory/astrofle-patch-1
vcatlett Oct 3, 2023
d185bf5
gettp working for scanlist
mpound Oct 10, 2023
8ab49e3
getting ps to work with scanlist
mpound Oct 12, 2023
9f15e52
ps and tp pytests passing
mpound Oct 12, 2023
e4ef12a
ignore notebooks
mpound Oct 12, 2023
5bab62e
fix weights to gbtidl style. oof
mpound Oct 13, 2023
958be69
new subbeammod passing test
mpound Oct 16, 2023
e8eb822
remove unused print
mpound Oct 16, 2023
9cbb41f
remove print statements, obsblock and obsblock docs
mpound Oct 16, 2023
144c44b
remove obsblock docs
mpound Oct 16, 2023
c16282f
add missing run
mpound Oct 16, 2023
bd8ec40
multifits files
mpound Oct 16, 2023
3ed5f00
Make calibrate keyword standard.
mpound Oct 17, 2023
e22d305
Added data set for multifits testing
astrofle Oct 30, 2023
c43f135
Added notebook for debugging multifits
astrofle Oct 30, 2023
7464939
Updated GBTFITSLoad to use verbose when loading multiple fits files
astrofle Oct 30, 2023
9791d2b
Updated and added testing notebooks for multifits and OffOn
astrofle Oct 31, 2023
830836a
Added notebook exploring the use of timeaverage when one or more inte…
astrofle Nov 1, 2023
6852268
Updated notebook for testing the handling of NaN integrations
astrofle Nov 1, 2023
abf6ad2
Added notebook using ScanBlock
astrofle Nov 1, 2023
1640ba1
fix issue #102
mpound Nov 3, 2023
1f32d72
testing multfits
mpound Nov 3, 2023
2e2598f
Merge branch 'multifits' of github.com:GreenBankObservatory/dysh into…
mpound Nov 3, 2023
cc34dcf
deal with PROC in onoffscanlist
mpound Nov 3, 2023
5838eb8
raise exception for empty scanblock
mpound Nov 3, 2023
609e45c
Added workflow_dispatch for running pytest
astrofle Nov 6, 2023
4ab6ca1
Fixed failing test
astrofle Nov 6, 2023
a1ab57a
Adding integration tests for loading multiple SDFITS files
astrofle Nov 6, 2023
89a06bc
Moved data files to testdata
astrofle Nov 6, 2023
298320e
Updated data path
astrofle Nov 6, 2023
e300687
Fixed merge conflicts
astrofle Nov 6, 2023
5fea564
Merge branch 'pedro-devel' of github.com:GreenBankObservatory/dysh in…
astrofle Nov 6, 2023
f1e62b1
Updated tests to work with ScanBlock
astrofle Nov 6, 2023
a0348a7
Moved test data to testdata
astrofle Nov 6, 2023
dd27237
Removed data under tests dir
astrofle Nov 6, 2023
354c449
Added conftest for spectra tests
astrofle Nov 6, 2023
2e7bdba
Updated list of test files expected
astrofle Nov 6, 2023
3efa1e2
remove nans in average
mpound Nov 6, 2023
a2550eb
Added test for multifits file using OffOn scans
astrofle Nov 6, 2023
a334da3
- handle averaging with blanked integrations (all NaN)
mpound Nov 8, 2023
c3b8114
get the weights right for all nan scans. also use where not argwhere!
mpound Nov 8, 2023
651853a
working now
mpound Nov 8, 2023
5db0825
working now
mpound Nov 8, 2023
603b38a
working now
mpound Nov 8, 2023
09f194f
working now
mpound Nov 8, 2023
c3597bf
working now
mpound Nov 8, 2023
4aa50ec
working now
mpound Nov 8, 2023
63ea84a
Merge branch 'multifits' of github.com:GreenBankObservatory/dysh into…
mpound Nov 8, 2023
0a39a6c
add get_project_testdata()
mpound Nov 9, 2023
225f0b1
make work with get_project_testdata()
mpound Nov 9, 2023
bfc038c
make work with get_project_testdata()
mpound Nov 9, 2023
4a7be0e
Merge from branch main
astrofle Nov 10, 2023
7961f04
Ran pre-commit hooks on these files
astrofle Nov 10, 2023
c3359a8
Merge from branch main
astrofle Nov 10, 2023
83a6624
Restored requirements
astrofle Nov 10, 2023
6e7f0db
Updated test for gbtfitsload
astrofle Nov 10, 2023
4c4d907
Remove indetation from docstrings to match Numpy style
astrofle Nov 10, 2023
6d797e7
Fixed: Title underline too short
astrofle Nov 10, 2023
7efe1b8
Updated system temperature calculations
astrofle Nov 10, 2023
65da03f
Added system temperature tests for scans
astrofle Nov 10, 2023
e0e0287
Added notebook used for trimming sdfits files so they can fit in test…
astrofle Nov 10, 2023
a1fb7f3
Updated comments on notebook
astrofle Nov 10, 2023
9769b3c
Updated indentation in docstrings
astrofle Nov 10, 2023
8d74445
Added instructions on how to create testdata/TGBT21A_501_11/NGC2782
astrofle Nov 10, 2023
8898123
Added test for individual exposure times for gettp
astrofle Nov 10, 2023
2365044
Updates testdata to use patched GBTIDL version
astrofle Nov 11, 2023
1d67e99
Added notes related to comparing to GBTIDL
astrofle Nov 11, 2023
6bde9e5
PSScan now tracks the exposure time when time averaging
astrofle Nov 11, 2023
0b14e02
Moved sq_weighted_avg from util to spectra
astrofle Nov 13, 2023
0f55369
Added data for testing getps with blanked integrations
astrofle Nov 13, 2023
ae60021
Modified PSScan to work with blanked integrations
astrofle Nov 13, 2023
aafbb78
Added test for getps with blanked integrations
astrofle Nov 13, 2023
745c4cc
Updated getps test with blanks
astrofle Nov 13, 2023
8f4abd2
Updated import of sq_weighted_avg
astrofle Nov 13, 2023
6cd539b
Updated test_timeaverage_with_NaNs.ipynb
astrofle Nov 13, 2023
7adc223
Updated data paths in notebooks
astrofle Nov 13, 2023
67af678
Added notebooks used to extract SDFITS file for testing with blanked …
astrofle Nov 13, 2023
da26dd5
Added notebooks used to test exposures in getps and gettp
astrofle Nov 13, 2023
08baa99
TPScan now uses float64 as data type instead of float32 and ignores b…
astrofle Nov 13, 2023
8f9a7fa
Moved test data for AGBT18B_354_03 -- OffOn observations
astrofle Nov 13, 2023
3461fd6
Updated hatch docs command to point to docs/source
astrofle Nov 13, 2023
9f55b15
Added a note about the size of the file used for the tutorial
astrofle Nov 13, 2023
7451f7a
add minimal failing example
mpound Nov 14, 2023
f42a338
fix failing getps (issue #102)
mpound Nov 14, 2023
ef3147c
Merge branch 'multifits' of github.com:GreenBankObservatory/dysh into…
mpound Nov 14, 2023
bf9361c
assignment got deleted -- did black do this??
mpound Nov 14, 2023
1d27cdd
Updated example notebook to work with ScanBlocks
astrofle Nov 15, 2023
1ef5f73
Updated example notebook to work with ScanBlocks
astrofle Nov 15, 2023
1882bfa
Merge branch 'multifits' of github.com:GreenBankObservatory/dysh into…
astrofle Nov 15, 2023
7a334ed
Added configuration for pytest to ignore DeprecationWarning
astrofle Nov 15, 2023
bb920e1
Test for OffOn calibration passing
astrofle Nov 15, 2023
4c099af
Updated tests
astrofle Nov 15, 2023
f48f185
Updated docstrings
astrofle Nov 15, 2023
3098c3a
remove debug statements
mpound Nov 15, 2023
d7db09c
Merge branch 'multifits' of github.com:GreenBankObservatory/dysh into…
mpound Nov 15, 2023
bcd59a6
Added missing test file
astrofle Nov 15, 2023
c60c462
Merge branch 'multifits' of github.com:GreenBankObservatory/dysh into…
astrofle Nov 15, 2023
04d247a
fix for issue #78, hide index in summary
mpound Nov 16, 2023
ea7fe61
Merge branch 'multifits' of github.com:GreenBankObservatory/dysh into…
mpound Nov 16, 2023
defa949
testing show_index
mpound Nov 16, 2023
fea99ab
ran pre-commit on all files
mpound Nov 16, 2023
f25783d
fix up excludes and command line args
mpound Nov 17, 2023
633c785
overwrite with multifits version
mpound Nov 17, 2023
2f2ef14
overwrite with multifits version
mpound Nov 17, 2023
e2c6a2d
overwrite with multifits version
mpound Nov 17, 2023
9f82e56
overwrite with multifits version
mpound Nov 17, 2023
e720873
overwrite with multifits version
mpound Nov 17, 2023
7831886
remove unused
mpound Nov 17, 2023
5e8e6fc
overwrite with multifits version
mpound Nov 17, 2023
07fe3c0
overwrite with multfits version (scanblock)
mpound Nov 17, 2023
7d3aa39
overwrite with multifits version
mpound Nov 17, 2023
fdf4889
overwrite with multifits version
mpound Nov 17, 2023
c9bf4e5
preferred version, hand merge with multifits
mpound Nov 17, 2023
12deeaf
make consistent with release_resolve_conflicts
mpound Nov 17, 2023
8c0dbfb
prefer multifits version
mpound Nov 17, 2023
79ebd94
prefer release version (documentation fixes)
mpound Nov 17, 2023
c1d93cb
tools removed an important import
mpound Nov 17, 2023
e893b2d
hand merge with multifits
mpound Nov 17, 2023
973c814
hand merge from release
mpound Nov 17, 2023
13b5af8
prefer release version
mpound Nov 17, 2023
e62eb2c
prefer release version
mpound Nov 20, 2023
e35eed4
prefer multifits version
mpound Nov 20, 2023
294219c
merged with multifits version
mpound Nov 20, 2023
26025a6
merged with release version
mpound Nov 20, 2023
f76e5ae
use mulfifits version
mpound Nov 20, 2023
31d99cf
use multifits version
mpound Nov 20, 2023
1f577f6
resolve conflicts
mpound Nov 20, 2023
9dd0834
resolve conflicts
mpound Nov 20, 2023
ed84873
updated file for pytest
mpound Nov 20, 2023
d40bf4d
restore deleted latex string
mpound Nov 20, 2023
1610f9a
fix assert so it passes Windows OS test suite
mpound Nov 20, 2023
416e391
remove python 3.8 as we no longer support it
mpound Nov 20, 2023
ae7bb82
support for 3.9+
mpound Nov 20, 2023
8838c5e
support for 3.9+
mpound Nov 20, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions .github/workflows/hatch-and-pytest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,10 @@ name: Build and Pytest

on:
push:
branches: [ "main", "release-0.2.0", "cat-devel", "mwp-devel", "pedro-devel" ]
branches: [ "main", "release-*", "cat-devel", "mwp-devel", "pedro-devel", "evan-devel" ]
pull_request:
branches: [ "main", "release-0.2.0", "cat-devel", "mwp-devel", "pedro-devel" ]
branches: [ "main", "release-*", "cat-devel", "mwp-devel", "pedro-devel", "evan-devel" ]
workflow_dispatch: # needed for "Run" button to show up in action
jobs:
build:
runs-on: ${{ matrix.os }}
Expand All @@ -17,7 +18,7 @@ jobs:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python-version: ["3.8", "3.9", "3.10", "3.11"]
python-version: ["3.9", "3.10", "3.11"]

steps:
- uses: actions/checkout@v3
Expand All @@ -28,6 +29,7 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install flake8 pytest
pip install -r requirements.txt
pip install -e .
- name: Build with hatch
Expand Down
20 changes: 20 additions & 0 deletions .github/workflows/pre-commit.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
name: pre-commit

on:
pull_request:
push:
branches: [main, release*]


jobs:
pre-commit:
env:
SKIP: ruff
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
# must match the version in .pre-commit-config.yaml:default_language_version.python
python-version: '3.9'
- uses: pre-commit/[email protected]
2 changes: 1 addition & 1 deletion .github/workflows/pyinstaller.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python-version: ["3.8", "3.9", "3.10", "3.11"]
python-version: ["3.9", "3.10", "3.11"]

steps:
- uses: actions/checkout@v3
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ on:
release:
types: [ published ]
workflow_dispatch:

jobs:
build:
runs-on: ${{ matrix.os }}
Expand Down
6 changes: 3 additions & 3 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ repos:
rev: v4.4.0
hooks:
- id: trailing-whitespace
exclude: '(notebooks|attic|benchmark|testdata)/.*'
- id: end-of-file-fixer
exclude: LICENSE
- id: check-yaml
Expand All @@ -24,13 +25,12 @@ repos:
rev: 5.12.0
hooks:
- id: isort
args: ["--profile", "black", "--filter-files"]
exclude: ^notebooks/
exclude: '(notebooks|attic|benchmark|testdata)/.*'
- repo: 'https://github.com/psf/black'
rev: 23.1.0
hooks:
- id: black
exclude: ^notebooks/
exclude: '(notebooks|attic|benchmark|testdata)/.*'
# - repo: https://github.com/jazzband/pip-tools
# rev: 6.12.3
# hooks:
Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
[![Documentation Status](https://readthedocs.org/projects/dysh/badge/?version=latest)](https://dysh.readthedocs.io/en/latest/?badge=latest)

# Dysh
# dysh

*Dysh* is a Python spectral line data reduction and analysis program for singledish data with specific emphasis on data from the Green Bank Telescope. It is currently under development in collaboration between the [Green Bank Observatory](https:/greenbankobservatory.org) and the Laboratory for Millimeter-Wave Astronomy (LMA) at [University of Maryland (UMD)](https://www.astro.umd.edu). It is intended to be a full replacement for the GBO's current reduction package [GBTIDL](https://www.gb.nrao.edu/GBT/DA/gbtidl/users_guide/).
*dysh* is a Python spectral line data reduction and analysis program for singledish data with specific emphasis on data from the Green Bank Telescope. It is currently under development in collaboration between the [Green Bank Observatory](https:/greenbankobservatory.org) and the Laboratory for Millimeter-Wave Astronomy (LMA) at [University of Maryland (UMD)](https://www.astro.umd.edu). It is intended to be a full replacement for the GBO's current reduction package [GBTIDL](https://www.gb.nrao.edu/GBT/DA/gbtidl/users_guide/).

## Getting Started
### Installation

Dysh requires Python 3.8+ and recent versions of [astropy]( https://astropy.org), [numpy](https://numpy.org), [scipy](https://scipy.org), [pandas](https://pandas.pydata.org), [specutils](https://specutils.readthedocs.io/en/stable/), and [matplotlib](https://matplotlib.org).
dysh requires Python 3.9+ and recent versions of [astropy]( https://astropy.org), [numpy](https://numpy.org), [scipy](https://scipy.org), [pandas](https://pandas.pydata.org), [specutils](https://specutils.readthedocs.io/en/stable/), and [matplotlib](https://matplotlib.org).

#### With pip from PyPi
Dysh is most easily installed with *pip*, which will take care of any dependencies. The packaged code is hosted at the [Python Packaging Index](https://pypi.org/project/dysh).
dysh is most easily installed with *pip*, which will take care of any dependencies. The packaged code is hosted at the [Python Packaging Index](https://pypi.org/project/dysh).

```bash
$ pip install dysh
Expand Down
2 changes: 1 addition & 1 deletion attic/sdfitsload.py
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ def dcmeantsys(calon, caloff, tcal, mode=0, fedge=10, nedge=None):
mode=1 do the mean after the division
"""
nchan = len(calon)
if nedge == None:
if nedge is None:
nedge = nchan // fedge # 10 %
if mode == 0:
meanoff = np.mean(caloff[nedge:-nedge])
Expand Down
4 changes: 2 additions & 2 deletions benchmark/revisedstructure.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@

import numpy as np
from astropy.table import Table, vstack
from specutils import SpectralRegion, Spectrum1D, SpectrumList
from specutils import SpectralRegion

from dysh.sdfitsload import Obsblock, SDFITSLoad, baseline, get_size
from dysh.sdfitsload import SDFITSLoad, baseline

if __name__ == "__main__":
parser = argparse.ArgumentParser(prog="revisedstructure")
Expand Down
25 changes: 10 additions & 15 deletions docs/source/design/sdfits_loaders.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,15 @@ Motivation

Although the initial design will be for GBT data, a goal is for `dysh` to be easily modifiable for any single-dish radio telescope. Thus came the idea of an SDFITS loader which would standardize inputs.

Obsblock
========
ScanBlock
=========

Here's the class diagram for an `Obsblock` and its derived classes.
Here's the class diagram for an `ScanBlock` and its derived classes.

.. mermaid::

classDiagram
class Obsblock{
class ScanBlock{
metadata
spectra
summary()
Expand All @@ -35,23 +35,18 @@ Here's the class diagram for an `Obsblock` and its derived classes.
spectra
fold()
}
class NodScan{
class SubBeamNodScan{
metadata
spectra
}
class OTFScan{
metadata
spectra
}
class ScanAvg{
metadata
spectra
}
Obsblock <|-- PSScan
Obsblock <|-- FSScan
Obsblock <|-- NodScan
Obsblock <|-- OTFScan
Obsblock <|-- ScanAvg
ScanBlock <|-- PSScan
ScanBlock <|-- FSScan
ScanBlock <|-- NodScan
ScanBlock <|-- OTFScan

That's probably not super accurate. I just copied the diagram in the stakeholder presentation from last May.

Expand All @@ -64,7 +59,7 @@ SDFITSLoad
class SDFITSLoad{
_filename
_bintable
_ptable
_index
_binheader
_data
_hdu
Expand Down
2 changes: 1 addition & 1 deletion docs/source/for_developers/git_workflows.rst
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ Releases
Release branches will be locked once work on the next release begins.

Setting up your own development branch
========
======================================

In the directory you want to work in, set up the repo:

Expand Down
2 changes: 1 addition & 1 deletion docs/source/getting_started/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
Installing ``dysh``
*******************

``dysh`` requires Python 3.8+ and recent versions of
``dysh`` requires Python 3.9+ and recent versions of
`astropy <https://astropy.org>`_,
`numpy <https://numpy.org>`_,
`scipy <https://scipy.org>`_,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,9 @@ In a `Python` instance, type
Keep the `Python` instance open, we will start working with `dysh` now.

.. note::
The data used for this tutorial is ~800 MB. Make sure you have enough disk space and bandwidth to download it.

Inspecting the raw data
-----------------------

Expand Down
7 changes: 0 additions & 7 deletions docs/source/modules/dysh.spectra.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,3 @@ Scan
:members:
:undoc-members:
:show-inheritance:

Obsblock
--------
.. automodule:: dysh.spectra.obsblock
:members:
:undoc-members:
:show-inheritance:
Loading