Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

233 ensure relevant notebooks have selection example #242

Merged
merged 22 commits into from
May 7, 2024

Conversation

mpound
Copy link
Collaborator

@mpound mpound commented May 6, 2024

This PR puts selection-based calibration examples in existing notebooks.
Also adds compound exclusion region baseline examplle to frequency switch notebook.
Also fixes a bug in getfs when using selection.

@mpound mpound linked an issue May 6, 2024 that may be closed by this pull request
@mpound mpound requested a review from astrofle May 6, 2024 17:18
@mpound
Copy link
Collaborator Author

mpound commented May 6, 2024

My bad. forgot to update a selection test. Will do now

@mpound
Copy link
Collaborator Author

mpound commented May 6, 2024

Should be good to go now @astrofle

@astrofle
Copy link
Collaborator

astrofle commented May 6, 2024

There's one error in the subbeamnod notebook:

ValueError                                Traceback (most recent call last)
Cell In[5], line 1
----> 1 sb3=sdfits.subbeamnod(method='scan')

File [/home/sandboxes/psalas/Dysh/dysh-fs/src/dysh/fits/gbtfitsload.py:1189](http://fourier.gb.nrao.edu:9030/home/sandboxes/psalas/Dysh/dysh-fs/src/dysh/fits/gbtfitsload.py#line=1188), in GBTFITSLoad.subbeamnod(self, method, sig, cal, calibrate, timeaverage, polaverage, weights, bintable, **kwargs)
   1186     reftp.append(tpoff[0])
   1187     # in order to reproduce gbtidl tsys, we need to do a normal
   1188     # total power scan
-> 1189     ftp = self.gettp(
   1190         scan=scan,
   1191         sig=None,
   1192         cal=None,
   1193         bintable=bintable,
   1194         fdnum=fn,
   1195         plnum=plnum,
   1196         ifnum=k,
   1197         weights=weights,
   1198         calibrate=calibrate,
   1199     )  # .timeaverage(weights=w)
   1200     fulltp.append(ftp[0])
   1201 sb = SubBeamNodScan(sigtp, reftp, fulltp, method=method, calibrate=calibrate, weights=weights)

File [/home/sandboxes/psalas/Dysh/dysh-fs/src/dysh/fits/gbtfitsload.py:925](http://fourier.gb.nrao.edu:9030/home/sandboxes/psalas/Dysh/dysh-fs/src/dysh/fits/gbtfitsload.py#line=924), in GBTFITSLoad.gettp(self, sig, cal, calibrate, timeaverage, polaverage, weights, bintable, **kwargs)
    923         kwargs[k] = v
    924 # now downselect with any additional kwargs
--> 925 ps_selection._select_from_mixed_kwargs(**kwargs)
    926 _sf = ps_selection.final
    927 ifnum = uniq(_sf["IFNUM"])

File [/home/sandboxes/psalas/Dysh/dysh-fs/src/dysh/util/selection.py:835](http://fourier.gb.nrao.edu:9030/home/sandboxes/psalas/Dysh/dysh-fs/src/dysh/util/selection.py#line=834), in Selection._select_from_mixed_kwargs(self, **kwargs)
    833 if debug:
    834     print(f"selection {ukwargs}")
--> 835 self.select(**ukwargs, tag=tag)

File [/home/sandboxes/psalas/Dysh/dysh-fs/src/dysh/util/selection.py:544](http://fourier.gb.nrao.edu:9030/home/sandboxes/psalas/Dysh/dysh-fs/src/dysh/util/selection.py#line=543), in Selection.select(self, tag, **kwargs)
    542     warnings.warn("Your selection rule resulted in no data being selected. Ignoring.")
    543     return
--> 544 self._addrow(row, df, tag)

File [/home/sandboxes/psalas/Dysh/dysh-fs/src/dysh/util/selection.py:461](http://fourier.gb.nrao.edu:9030/home/sandboxes/psalas/Dysh/dysh-fs/src/dysh/util/selection.py#line=460), in Selection._addrow(self, row, dataframe, tag)
    442 def _addrow(self, row, dataframe, tag=None):
    443     """
    444     Common code to add a tagged row to the internal table after the selection has been created.
    445     Should be called in select* methods.
   (...)
    459 
    460     """
--> 461     if self._check_for_duplicates(dataframe):
    462         return
    463     if tag is not None:

File [/home/sandboxes/psalas/Dysh/dysh-fs/src/dysh/util/selection.py:432](http://fourier.gb.nrao.edu:9030/home/sandboxes/psalas/Dysh/dysh-fs/src/dysh/util/selection.py#line=431), in Selection._check_for_duplicates(self, df)
    429 for _id, s in self._selection_rules.items():
    430     if s.equals(df):
    431         # print(s, df)
--> 432         tag = self._table.loc[_id]["TAG"]
    433         # raise Exception(
    434         warnings.warn(
    435             f"A rule that results in an identical selection has already been added: ID: {_id}, TAG:{tag}."
    436             " Ignoring."
    437         )

File [/home/sandboxes/psalas/Dysh/dysh-fs/dysh-py3.11/lib/python3.11/site-packages/astropy/table/index.py:876](http://fourier.gb.nrao.edu:9030/home/sandboxes/psalas/Dysh/dysh-fs/dysh-py3.11/lib/python3.11/site-packages/astropy/table/index.py#line=875), in TableLoc.__getitem__(self, item)
    863 def __getitem__(self, item):
    864     """
    865     Retrieve Table rows by value slice.
    866 
   (...)
    874         second element must be as above.
    875     """
--> 876     rows = self._get_rows(item)
    878     if len(rows) == 0:  # no matches found
    879         raise KeyError(f"No matches found for key {item}")

File [/home/sandboxes/psalas/Dysh/dysh-fs/dysh-py3.11/lib/python3.11/site-packages/astropy/table/index.py:834](http://fourier.gb.nrao.edu:9030/home/sandboxes/psalas/Dysh/dysh-fs/dysh-py3.11/lib/python3.11/site-packages/astropy/table/index.py#line=833), in TableLoc._get_rows(self, item)
    830 """
    831 Retrieve Table rows indexes by value slice.
    832 """
    833 if len(self.indices) == 0:
--> 834     raise ValueError("Can only use TableLoc for a table with indices")
    836 if isinstance(item, tuple):
    837     key, item = item

ValueError: Can only use TableLoc for a table with indices

@astrofle
Copy link
Collaborator

astrofle commented May 7, 2024

There's one error in the subbeamnod notebook:

ValueError                                Traceback (most recent call last)
Cell In[5], line 1
----> 1 sb3=sdfits.subbeamnod(method='scan')

File [/home/sandboxes/psalas/Dysh/dysh-fs/src/dysh/fits/gbtfitsload.py:1189](http://fourier.gb.nrao.edu:9030/home/sandboxes/psalas/Dysh/dysh-fs/src/dysh/fits/gbtfitsload.py#line=1188), in GBTFITSLoad.subbeamnod(self, method, sig, cal, calibrate, timeaverage, polaverage, weights, bintable, **kwargs)
   1186     reftp.append(tpoff[0])
   1187     # in order to reproduce gbtidl tsys, we need to do a normal
   1188     # total power scan
-> 1189     ftp = self.gettp(
   1190         scan=scan,
   1191         sig=None,
   1192         cal=None,
   1193         bintable=bintable,
   1194         fdnum=fn,
   1195         plnum=plnum,
   1196         ifnum=k,
   1197         weights=weights,
   1198         calibrate=calibrate,
   1199     )  # .timeaverage(weights=w)
   1200     fulltp.append(ftp[0])
   1201 sb = SubBeamNodScan(sigtp, reftp, fulltp, method=method, calibrate=calibrate, weights=weights)

File [/home/sandboxes/psalas/Dysh/dysh-fs/src/dysh/fits/gbtfitsload.py:925](http://fourier.gb.nrao.edu:9030/home/sandboxes/psalas/Dysh/dysh-fs/src/dysh/fits/gbtfitsload.py#line=924), in GBTFITSLoad.gettp(self, sig, cal, calibrate, timeaverage, polaverage, weights, bintable, **kwargs)
    923         kwargs[k] = v
    924 # now downselect with any additional kwargs
--> 925 ps_selection._select_from_mixed_kwargs(**kwargs)
    926 _sf = ps_selection.final
    927 ifnum = uniq(_sf["IFNUM"])

File [/home/sandboxes/psalas/Dysh/dysh-fs/src/dysh/util/selection.py:835](http://fourier.gb.nrao.edu:9030/home/sandboxes/psalas/Dysh/dysh-fs/src/dysh/util/selection.py#line=834), in Selection._select_from_mixed_kwargs(self, **kwargs)
    833 if debug:
    834     print(f"selection {ukwargs}")
--> 835 self.select(**ukwargs, tag=tag)

File [/home/sandboxes/psalas/Dysh/dysh-fs/src/dysh/util/selection.py:544](http://fourier.gb.nrao.edu:9030/home/sandboxes/psalas/Dysh/dysh-fs/src/dysh/util/selection.py#line=543), in Selection.select(self, tag, **kwargs)
    542     warnings.warn("Your selection rule resulted in no data being selected. Ignoring.")
    543     return
--> 544 self._addrow(row, df, tag)

File [/home/sandboxes/psalas/Dysh/dysh-fs/src/dysh/util/selection.py:461](http://fourier.gb.nrao.edu:9030/home/sandboxes/psalas/Dysh/dysh-fs/src/dysh/util/selection.py#line=460), in Selection._addrow(self, row, dataframe, tag)
    442 def _addrow(self, row, dataframe, tag=None):
    443     """
    444     Common code to add a tagged row to the internal table after the selection has been created.
    445     Should be called in select* methods.
   (...)
    459 
    460     """
--> 461     if self._check_for_duplicates(dataframe):
    462         return
    463     if tag is not None:

File [/home/sandboxes/psalas/Dysh/dysh-fs/src/dysh/util/selection.py:432](http://fourier.gb.nrao.edu:9030/home/sandboxes/psalas/Dysh/dysh-fs/src/dysh/util/selection.py#line=431), in Selection._check_for_duplicates(self, df)
    429 for _id, s in self._selection_rules.items():
    430     if s.equals(df):
    431         # print(s, df)
--> 432         tag = self._table.loc[_id]["TAG"]
    433         # raise Exception(
    434         warnings.warn(
    435             f"A rule that results in an identical selection has already been added: ID: {_id}, TAG:{tag}."
    436             " Ignoring."
    437         )

File [/home/sandboxes/psalas/Dysh/dysh-fs/dysh-py3.11/lib/python3.11/site-packages/astropy/table/index.py:876](http://fourier.gb.nrao.edu:9030/home/sandboxes/psalas/Dysh/dysh-fs/dysh-py3.11/lib/python3.11/site-packages/astropy/table/index.py#line=875), in TableLoc.__getitem__(self, item)
    863 def __getitem__(self, item):
    864     """
    865     Retrieve Table rows by value slice.
    866 
   (...)
    874         second element must be as above.
    875     """
--> 876     rows = self._get_rows(item)
    878     if len(rows) == 0:  # no matches found
    879         raise KeyError(f"No matches found for key {item}")

File [/home/sandboxes/psalas/Dysh/dysh-fs/dysh-py3.11/lib/python3.11/site-packages/astropy/table/index.py:834](http://fourier.gb.nrao.edu:9030/home/sandboxes/psalas/Dysh/dysh-fs/dysh-py3.11/lib/python3.11/site-packages/astropy/table/index.py#line=833), in TableLoc._get_rows(self, item)
    830 """
    831 Retrieve Table rows indexes by value slice.
    832 """
    833 if len(self.indices) == 0:
--> 834     raise ValueError("Can only use TableLoc for a table with indices")
    836 if isinstance(item, tuple):
    837     key, item = item

ValueError: Can only use TableLoc for a table with indices

The error shows up when using astropy==6.1.0. It does not show for astropy==6.0.0.

@mpound
Copy link
Collaborator Author

mpound commented May 7, 2024

astropy pinned to <6.1. FS notebook changed to use wget

Copy link
Collaborator

@astrofle astrofle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need an issue to track the astropy version cap.

src/dysh/spectra/spectrum.py Outdated Show resolved Hide resolved
@mpound mpound merged commit 68d116b into main May 7, 2024
14 checks passed
@mpound mpound deleted the 233-ensure-relevant-notebooks-have-selection-example branch May 7, 2024 18:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ensure relevant notebooks have selection example
2 participants