Skip to content

Commit

Permalink
Merge branch 'main' into shower_flagging_enhancement
Browse files Browse the repository at this point in the history
  • Loading branch information
mwregan2 authored Mar 11, 2024
2 parents 747c652 + 4ddddf8 commit fc7fe3f
Show file tree
Hide file tree
Showing 13 changed files with 511 additions and 241 deletions.
70 changes: 65 additions & 5 deletions CHANGES.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,64 @@
1.5.3 (unreleased)
1.6.2 (unreleased)
==================

-
Changes to API
--------------

Bug Fixes
---------

Other
-----

1.6.1 (2024-02-29)
==================

Changes to API
--------------

ramp_fitting
~~~~~~~~~~~~

- Add ``average_dark_current`` to calculations of poisson variance. [#243]

1.6.0 (2024-02-15)
==================

Changes to API
--------------

jump
~~~~

- Add in the flagging of groups in the integration after a snowball
occurs. The saturated core of the snowball gets flagged as jump
for a number of groups passed in as a parameter [#238]

Bug Fixes
---------

jump
~~~~

- Fixed the computation of the number of rows per slice for multiprocessing, which
was causing different results when running the step with multiprocess [#239]

- Fix the code to at least always flag the group with the shower and the requested
groups after the primary shower. [#237]

Other
-----

jump
~~~~

- Reorganize jump docs between the jwst and stcal repos. [#240]

ramp_fitting
~~~~~~~~~~~~

- Reorganize ramp_fitting docs between the jwst and stcal repos. [#240]


1.5.2 (2023-12-13)
==================
Expand All @@ -24,7 +81,7 @@ Other
- Enable automatic linting and code style checks [#187]

ramp_fitting
------------
~~~~~~~~~~~~

- Refactor Casertano, et.al, 2022 uneven ramp fitting and incorporate the matching
jump detection algorithm into it. [#215]
Expand Down Expand Up @@ -84,13 +141,16 @@ jump
within a group. [#207]

- Added more allowable selections for the number of cores to use for
multiprocessing [#183].
multiprocessing [#183]

- Fixed the computation of the number of rows per slice for multiprocessing,
which caused different results when running the step with multiprocess [#239]

ramp_fitting
~~~~~~~~~~~~

- Added more allowable selections for the number of cores to use for
multiprocessing [#183].
multiprocessing [#183]

- Updating variance computation for invalid integrations, as well as
updating the median rate computation by excluding groups marked as
Expand Down
55 changes: 31 additions & 24 deletions docs/stcal/jump/description.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
.. _jump_algorithm:

Algorithm
---------
This routine detects jumps in an exposure by looking for outliers
This routine detects jumps by looking for outliers
in the up-the-ramp signal for each pixel in each integration within
an input exposure. On output, the GROUPDQ array is updated with the DQ flag
"JUMP_DET" to indicate the location of each jump that was found.
Expand All @@ -10,34 +12,39 @@ output PIXELDQ array.
The SCI and ERR arrays of the input data are not modified.

The current implementation uses the two-point difference method described
in Anderson&Gordon2011_.
in `Anderson & Gordon (2011) <https://ui.adsabs.harvard.edu/abs/2011PASP..123.1237A>`_.

Two-Point Difference Method
^^^^^^^^^^^^^^^^^^^^^^^^^^^
The two-point difference method is applied to each integration as follows:

* Compute the first differences for each pixel (the difference between
adjacent groups)
* Compute the clipped (dropping the largest difference) median of the first differences for each pixel.
* Use the median to estimate the Poisson noise for each group and combine it
with the read noise to arrive at an estimate of the total expected noise for
each difference.
* Compute the "difference ratio" as the difference between the first differences
of each group and the median, divided by the expected noise.
* If the largest "difference ratio" is greater than the rejection threshold,
flag the group corresponding to that ratio as having a jump.
* If a jump is found in a given pixel, iterate the above steps with the
jump-impacted group excluded, looking for additional lower-level jumps
that still exceed the rejection threshold.
* Stop iterating on a given pixel when no new jumps are found or only one
difference remains.
* If the there are only three differences (four groups), the standard median
is used rather than the clipped median.
* If there are only two differences (three groups), the smallest one is compared to the larger
one and if the larger one is above a threshold, it is flagged as a jump.
#. Compute the first differences for each pixel (the difference between
adjacent groups)
#. Compute the clipped median (dropping the largest difference) of the first differences for each pixel.
If there are only three first difference values (four groups), no clipping is
performed when computing the median.
#. Use the median to estimate the Poisson noise for each group and combine it
with the read noise to arrive at an estimate of the total expected noise for
each difference.
#. Compute the "difference ratio" as the difference between the first differences
of each group and the median, divided by the expected noise.
#. If the largest "difference ratio" is greater than the rejection threshold,
flag the group corresponding to that ratio as having a jump.
#. If a jump is found in a given pixel, iterate the above steps with the
jump-impacted group excluded, looking for additional lower-level jumps
that still exceed the rejection threshold.
#. Stop iterating on a given pixel when no new jumps are found or only one
difference remains.
#. If there are only two differences (three groups), the smallest one is compared to the larger
one and if the larger one is above a threshold, it is flagged as a jump.
#. If flagging of the 4 neighbors is requested, then the 4 adjacent pixels will
have ramp jumps flagged in the same group as the central pixel as long as it has
a jump between the min and max requested levels for this option.
#. If flagging of groups after a ramp jump is requested, then the groups in the
requested time since a detected ramp jump will be flagged as ramp jumps if
the ramp jump is above the requested threshold. Two thresholds and times are
possible for this option.

Note that any ramp values flagged as SATURATED in the input GROUPDQ array
Note that any ramp groups flagged as SATURATED in the input GROUPDQ array
are not used in any of the above calculations and hence will never be
marked as containing a jump.

.. _Anderson&Gordon2011: https://ui.adsabs.harvard.edu/abs/2011PASP..123.1237A
Loading

0 comments on commit fc7fe3f

Please sign in to comment.