Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

developing back end run methods #216

Merged
merged 31 commits into from
Dec 12, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
e71456d
developing back end run methods
cadeduckworth Sep 15, 2022
6c67ec1
condensed backend run methods
cadeduckworth Sep 15, 2022
7bcdf49
Addressed change requests by Oliver
cadeduckworth Oct 22, 2022
a87cb11
fixed indentation error for _single_frame() block
cadeduckworth Oct 22, 2022
7edde0d
reverting previous tab issue, necessary for normal function
cadeduckworth Oct 22, 2022
48016c5
fixed errors previously reverted for testing
cadeduckworth Oct 23, 2022
59c819c
Merge branch 'develop' into ensemble_run_update
cadeduckworth Oct 23, 2022
fb58f27
doc string format/syntax issue
cadeduckworth Oct 25, 2022
1f7f2d1
checkout from alt repository
cadeduckworth Oct 25, 2022
179603f
changed base class function definitions from pass to raise NotImpleme…
cadeduckworth Oct 28, 2022
46d3cf2
Merge branch 'ensemble_run_update' of github.com:Becksteinlab/MDPOW i…
cadeduckworth Oct 28, 2022
f47c806
adding preliminary test for try-except pattern addition to backend ru…
cadeduckworth Nov 2, 2022
5e02322
removed # pragma: no cover from _single_frame() and _single_universe()
cadeduckworth Nov 4, 2022
2fc9135
changed dihedral atom group selection strings in test_dihedral.py to …
cadeduckworth Nov 4, 2022
37cd1d7
changed dihedral atom group selection order in test_dataframe to matc…
cadeduckworth Nov 4, 2022
0dc7e32
several changes and updates to test_dihedral.py, resulting in passing…
cadeduckworth Nov 12, 2022
a0de2d9
relocating test to external branch, 214_dihedral issue
cadeduckworth Nov 12, 2022
273c1af
moved raise NotImplementedError test for _single_universe() for Dihed…
cadeduckworth Nov 12, 2022
8212d11
Merge remote-tracking branch 'origin/214_dihedralanalysis-test-issues…
cadeduckworth Nov 12, 2022
653210e
added explicit backend run method tests for EnsembleAnalysis
cadeduckworth Nov 17, 2022
7eff11f
test_dihedral.py reflects same changes as PR#218
cadeduckworth Nov 17, 2022
67261cb
Merge branch 'develop' into ensemble_run_update
orbeckst Nov 17, 2022
2f63c3a
Merge branch 'develop' into ensemble_run_update
orbeckst Nov 17, 2022
ce02b3b
Merge branch 'develop' into ensemble_run_update
orbeckst Nov 18, 2022
cf4bd40
updated source and html docs and CHANGELOG to reflect changes made fo…
cadeduckworth Dec 8, 2022
f45fc6b
merge with origin changes
cadeduckworth Dec 8, 2022
1a90ed8
split EnsembleAnalysis run method test, one test for _single_frame, o…
cadeduckworth Dec 8, 2022
7054bb5
minor change to run method tests for proper use of example EnsembleAn…
cadeduckworth Dec 8, 2022
19f7c84
added and corrected use of sphinx markup for documentation, edited ve…
cadeduckworth Dec 8, 2022
40381aa
changes to docs for sphinx markup
cadeduckworth Dec 8, 2022
8870e39
whitespace and docstring formatting
cadeduckworth Dec 11, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions CHANGES
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,19 @@ CHANGES for MDPOW
Add summary of changes for each release. Use ISO 8061 dates. Reference
GitHub issues numbers and PR numbers.



2022-??-?? 0.8.1
2022-??-?? 0.9.0
cadeduckworth, orbeckst, VOD555

Changes

* for ensemble.EnsembleAnalysis._single_frame()
changed 'pass' to 'raise NotImplementedError' (#216)
* for ensemble.EnsembleAnalysis._single_universe()
changed 'pass' to 'raise NotImplementedError' (#216)
* for ensemble.EnsembleAnalysis.run() changed to detect
and use _single_frame OR _single_universe (#216)
* _prepare_universe and _conclude_universe removed from
EnsembleAnalysis.run() method, no longer needed (per comments, #199)
* added support for Python 3.10
* dropped testing on Python 3.6

Expand Down
12 changes: 10 additions & 2 deletions doc/sphinx/source/analysis/ensemble_analysis.txt
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,18 @@ Universes and be analyzed as a group.

:class:`~mdpow.analysis.ensemble.EnsembleAnalysis` is a class inspired by the
:class:`AnalysisBase <MDAnalysis.analysis.base.AnalysisBase>` from MDAnalysis which
iterates over the systems in the ensemble and the frames in the systems. It sets up both iterations between
universes and universe frames allowing for analysis to be run on both whole systems and the frames of those
iterates over the systems in the ensemble or the frames in the systems. It sets up iterations between
universes or universe frames allowing for analysis to be run on either whole systems or the frames of those
systems. This allows for users to easily run analyses on MDPOW simulations.

:exc:`NotImplementedError` will detect whether :meth:`~EnsembleAnalysis._single_universe`
or :meth:`~EnsembleAnalysis._single_frame` should be implemented, based on which
is defined in the :class:`~mdpow.analysis.ensemble.EnsembleAnalysis`. Only one of the
two methods should be defined for an :class:`~mdpow.analysis.ensemble.EnsembleAnalysis`.
For verbose functionality, the analysis may show two iteration bars,
where only one of which will actually be iterated, while the other will
load to completion instantaneously, showing the system that is being worked on.

.. autoclass:: mdpow.analysis.ensemble.EnsembleAnalysis
:members:

Expand Down
4 changes: 2 additions & 2 deletions mdpow/analysis/dihedral.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,8 +78,8 @@ def check_dihedral_inputs(selections):
for group in selections:
for k in group.keys():
if len(group[k]) != 4:
msg = ''''Dihedral calculations require AtomGroups with
only 4 atoms, %s selected''' % len(group)
msg = ("Dihedral calculations require AtomGroups with "
f"only 4 atoms, {len(group)} selected")
logger.error(msg)
raise SelectionError(msg)

Expand Down
64 changes: 42 additions & 22 deletions mdpow/analysis/ensemble.py
Original file line number Diff line number Diff line change
Expand Up @@ -464,19 +464,35 @@ def _setup_frames(self, trajectory):
self.times = np.zeros(self.n_frames)

def _single_universe(self):
"""Calculations on a single Universe object.

Run on each universe in the ensemble during when
self.run in called.
"""Calculations on a single :class:`MDAnalysis.Universe
<MDAnalysis.core.groups.universe.Universe>` object.

Run on each :class:`MDAnalysis.Universe
<MDAnalysis.core.groups.universe.Universe>`
in the :class:`~mdpow.analysis.ensemble.Ensemble`
during when :meth:`run` in called.
:exc:`NotImplementedError` will detect whether
:meth:`~EnsembleAnalysis._single_universe`
or :meth:`~EnsembleAnalysis._single_frame`
should be implemented, based on which is defined
in the :class:`~mdpow.analysis.ensemble.EnsembleAnalysis`.
"""
pass # pragma: no cover
raise NotImplementedError

def _single_frame(self):
"""Calculate data from a single frame of trajectory

Called on each frame for universes in the Ensemble.
"""Calculate data from a single frame of trajectory.

Called on each frame for each
:class:`MDAnalysis.Universe <MDAnalysis.core.groups.universe.Universe>`
in the :class:`~mdpow.analysis.ensemble.Ensemble`.

:exc:`NotImplementedError` will detect whether
:meth:`~EnsembleAnalysis._single_universe`
or :meth:`~EnsembleAnalysis._single_frame`
should be implemented, based on which is defined
in the :class:`~mdpow.analysis.ensemble.EnsembleAnalysis`.
"""
pass # pragma: no cover
raise NotImplementedError
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add test that checks that this NotImplementedError is raised. If necessary by calling the method explicitly

def test_single_frame_raises_NotImplementedError():
    ...
    ea = EnsembleAnalysis(...)
    with pytest.raises(NotImplementedError):
          ea._single_frame()

Copy link
Contributor Author

@cadeduckworth cadeduckworth Nov 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/Becksteinlab/MDPOW/blob/ensemble_run_update/mdpow/tests/test_dihedral.py

  • file does not exist in pull request because it was deleted and moved to another pull request, so I just linked the source from the branch
  • function is at end of module
  • this is the explicit call of _single_universe() to test NotImplementedError is raised when not defined, but specifically moved for use in DihedralAnalysis as instructed

I am going to add explicit tests for raising NotImplementedError for _single_frame() and _single_universe() similar to pattern in your comment above, placed in test_ensemble.py, because test_run.py is for the partition coefficient calculation and test_ensemble.py deals with EnsembleAnalysis class where this run method that calls single_frame/uni is defined

  • will be in this pull request

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look into https://github.com/Becksteinlab/MDPOW/pull/216/files and you see the annotation by codecov that says that the line was not covered by tests.

Also check the codecov report in the checks and ultimately https://app.codecov.io/gh/Becksteinlab/MDPOW/pull/216

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your tests do not check that the base class raises NotImplementedError.


def _prepare_ensemble(self):
"""For establishing data structures used in running
Expand Down Expand Up @@ -505,27 +521,31 @@ def _conclude_ensemble(self):
pass # pragma: no cover

def run(self, start=None, stop=None, step=None):
"""Runs _single_universe on each system and _single_frame
"""Runs :meth:`~EnsembleAnalysis._single_universe`
on each system or :meth:`~EnsembleAnalysis._single_frame`
on each frame in the system.

First iterates through keys of ensemble, then runs _setup_system
which defines the system and trajectory. Then iterates over
trajectory frames.
First iterates through keys of ensemble, then runs
:meth:`~EnsembleAnalysis._setup_system`which defines
the system and trajectory. Then iterates over each
system universe or trajectory frames of each universe
as defined by :meth:`~EnsembleAnalysis._single_universe`
or :meth:`~EnsembleAnalysis._single_frame`.
"""
logger.info("Setting up systems")
self._prepare_ensemble()
for self._key in ProgressBar(self._ensemble.keys(), verbose=True):
self._setup_system(self._key, start=start, stop=stop, step=step)
self._prepare_universe()
self._single_universe()
for i, ts in enumerate(ProgressBar(self._trajectory[self.start:self.stop:self.step], verbose=True,
try:
self._single_universe()
except NotImplementedError:
orbeckst marked this conversation as resolved.
Show resolved Hide resolved
for i, ts in enumerate(ProgressBar(self._trajectory[self.start:self.stop:self.step], verbose=True,
postfix=f'running system {self._key}')):
self._frame_index = i
self._ts = ts
self.frames[i] = ts.frame
self.times[i] = ts.time
self._single_frame()
self._conclude_universe()
self._frame_index = i
self._ts = ts
self.frames[i] = ts.frame
self.times[i] = ts.time
self._single_frame()
logger.info("Moving to next universe")
logger.info("Finishing up")
self._conclude_ensemble()
Expand Down
5 changes: 5 additions & 0 deletions mdpow/tests/test_dihedral.py
Original file line number Diff line number Diff line change
Expand Up @@ -93,3 +93,8 @@ def test_ValueError_different_ensemble(self):
match='Dihedral selections from different Ensembles, '):
DihedralAnalysis([dh1, dh2])

def test_single_universe(self):
dh = self.Ens.select_atoms('name C4', 'name C17', 'name S2', 'name N3')
with pytest.raises(NotImplementedError):
DihedralAnalysis([dh])._single_universe()

Comment on lines +96 to +100
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is not strictly necessary but we can leave it in, as a check that the DihedralAnalysis does not do something weird to _single_universe().

32 changes: 32 additions & 0 deletions mdpow/tests/test_ensemble.py
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,38 @@ def _conclude_universe(self):
TestRun = TestAnalysis(Sim).run(start=0, step=1, stop=10)
assert Sim.keys() == TestRun.key_list

def test_ensemble_analysis_run_frame(self):
class TestAnalysis(EnsembleAnalysis):
def __init__(self, test_ensemble):
super(TestAnalysis, self).__init__(test_ensemble)

self._ens = test_ensemble

def _single_universe(self):
pass

Sim = Ensemble(dirname=self.tmpdir.name, solvents=['water'])
TestRun = TestAnalysis(Sim)

with pytest.raises(NotImplementedError):
TestRun._single_frame()

def test_ensemble_analysis_run_universe(self):
class TestAnalysis(EnsembleAnalysis):
def __init__(self, test_ensemble):
super(TestAnalysis, self).__init__(test_ensemble)

self._ens = test_ensemble

def _single_frame(self):
pass

Sim = Ensemble(dirname=self.tmpdir.name, solvents=['water'])
TestRun = TestAnalysis(Sim)

with pytest.raises(NotImplementedError):
TestRun._single_universe()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#216 #216 (comment)

@orbeckst Can you provide more detail for the linked comment? This test might not have shown up with the recent commit, but if it did, can you elaborate on how it is not correctly testing the base class?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codecov indicates that these lines were not tested. Codecov is not always right but it's something to look into carefully. Come up with a way to convince yourself (and me) that your tests really test the code that you think they do. Perhaps temporarily change the raise NotImplementError to ValueError and then your test should fail. Report back on what you did.

Copy link
Contributor Author

@cadeduckworth cadeduckworth Dec 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed exception/error types and nothing changed. If the test and class definitions were nested correctly, pytest would have also picked up that
assert excinfo.type == should have been assert excinfo.type is, so basically nothing was happening. There is a way to match directly to string but I did it incorrectly and this way is simpler.

I split the single test into two separate, one for _singe_frame and one for _single_universe. Each has its own TestAnalysis class definition, where it shows that the run method not being tested is defined, so the undefined run method is explicitly tested for raising NotImplementedError, for each case.

There is probably a way to consolidate the two tests back into one, but it is very clear what is being done this way. It still seems there is a simpler method. I will review again in the morning and make changes requested for documentation so this can be merged if everything is correct otherwise!

def test_value_error(self):
ens = Ensemble(dirname=self.tmpdir.name, solvents=['water'])
copy_ens = Ensemble()
Expand Down