Upgrade packages post 2.8 release

[MikeSullivan7]

Issue

Closes #2268, progresses #2267.

Description

This PR updates the following packages used in MI:

astropy = 5.0.* -> 6.0.*
scipy = 1.8.* -> 1.14.*
numpy = 1.23.* -> 1.26.*
algotom = 1.0.* -> 1.6.*
cudatoolkit = 10.2.* -> 11.8.*
cupy = 10.2.* -> 12.3.*
jenkspy = 0.2.0 = 0.4.*

Testing

make check

Acceptance Criteria

Check that you can rebuild the dev environment without any conflicts with
python ./setup.py create_dev_env
and then MI runs as normal.

NOTE: There will be some depreciation warnings related to numpy, these will be dealt with in a future PR.
Also, some packages have been updated even though they were not in the original scope as there were conflicts with the dependencies of the packages we were trying to upgrade, e.g. numpy and scipy

Documentation

Will add release note

[coveralls]

coverage: 73.097% (-0.02%) from 73.115%
when pulling e9802cf on upgrade_packages_post_2.8_release
into 8cf298f on main.

[samtygier-stfc]

On IDAaaS and locally on linux I get a failure in the system test TestGuiSystemReconstruction.test_refine. I get the same error when i press the Minimise Error button in the recon window. Odd that this passes on github actions.

ERROR    mantidimaging.gui.dialogs.async_task.task:task.py:53 Failed to execute task: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
Traceback (most recent call last):
  File "/home/sam/git/mantidimaging/mantidimaging/gui/dialogs/async_task/task.py", line 50, in run
    self.result = call_with_known_parameters(self.task_function, **self.kwargs)
  File "/home/sam/git/mantidimaging/mantidimaging/core/utility/func_call.py", line 12, in call_with_known_parameters
    return func(**ka)
  File "/home/sam/git/mantidimaging/mantidimaging/gui/windows/recon/model.py", line 249, in auto_find_minimisation_sqsum
    cor = reconstructor.find_cor(self.images, slice, initial_cor[idx], recon_params)
  File "/home/sam/git/mantidimaging/mantidimaging/core/reconstruct/astra_recon.py", line 102, in find_cor
    return minimize(minimizer_function, start_cor, method='nelder-mead', tol=0.1).x[0]
  File "/home/sam/mambaforge/envs/mantidimaging-dev/lib/python3.10/site-packages/scipy/optimize/_minimize.py", line 719, in minimize
    res = _minimize_neldermead(fun, x0, args, callback, bounds=bounds,
  File "/home/sam/mambaforge/envs/mantidimaging-dev/lib/python3.10/site-packages/scipy/optimize/_optimize.py", line 817, in _minimize_neldermead
    fsim[k] = func(sim[k])
  File "/home/sam/mambaforge/envs/mantidimaging-dev/lib/python3.10/site-packages/scipy/optimize/_optimize.py", line 526, in function_wrapper
    fx = function(np.copy(x), *(wrapper_args + args))
  File "/home/sam/git/mantidimaging/mantidimaging/core/reconstruct/astra_recon.py", line 100, in minimizer_function
    return -get_sumsq(AstraRecon.single_sino(images.sino(slice_idx), ScalarCoR(cor), proj_angles, recon_params))
  File "/home/sam/git/mantidimaging/mantidimaging/core/reconstruct/astra_recon.py", line 118, in single_sino
    vectors = vec_geom_init2d(proj_angles, 1.0, cor.to_vec(image_width).value)
  File "/home/sam/git/mantidimaging/mantidimaging/core/reconstruct/astra_recon.py", line 39, in vec_geom_init2d
    vectors[i, 2:4] = np.dot(rotation_matrix2d(theta), d0)[:]  # center of detector position
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
Traceback (most recent call last):
  File "/home/sam/git/mantidimaging/mantidimaging/gui/test/gui_system_reconstruction_test.py", line 103, in <lambda>
    QTimer.singleShot(SHORT_DELAY, lambda: self._click_cor_inspect())
  File "/home/sam/git/mantidimaging/mantidimaging/gui/test/gui_system_reconstruction_test.py", line 70, in _click_cor_inspect
    cls._wait_for_widget_visible(CORInspectionDialogView)
  File "/home/sam/git/mantidimaging/mantidimaging/gui/test/gui_system_base.py", line 98, in _wait_for_widget_visible
    raise RuntimeError("_wait_for_stack_selector reach max retries")
RuntimeError: _wait_for_stack_selector reach max retries
XFAIL (Unresolved, see #1641)

I'll see if I can work out what is causing this.

[samtygier-stfc]

The error only happens locally because its being called from astra_recon.py on github actions it will be using the tomopy version because there is no GPU.

[samtygier-stfc]

Looks like it is the d0 line in vec_geom_init2d()

        vectors[i, 0:2] = np.dot(rotation_matrix2d(theta), s0)[:]  # ray position
        vectors[i, 2:4] = np.dot(rotation_matrix2d(theta), d0)[:]  # center of detector position
        vectors[i, 4:6] = np.dot(rotation_matrix2d(theta), u0)[:]  # detector pixel (0,0) to (0,1).

theta=0.0 s0=[0.0, -1.0] d0=[array([0.]), 0.0] u0=[1.0, 0.0]

[MikeSullivan7]

After running the systems tests on windows, I'm getting the following failures:

NOTE : TestGuiSystemLoading.test_load_with_start_stop_inc_0 and TestGuiSystemWindows.test_loaded_data are also failing in Main on Windows

============================================================ FAILURES =============================================================
______________________________________ TestGuiSystemLoading.test_load_with_start_stop_inc_0 _______________________________________

a = (<mantidimaging.gui.test.gui_system_loading_test.TestGuiSystemLoading testMethod=test_load_with_start_stop_inc_0>,), kw = {}

    @wraps(func)
    def standalone_func(*a, **kw):
>       return func(*(a + p.args), **p.kwargs, **kw)

..\AppData\Local\miniforge3\envs\mantidimaging-dev\lib\site-packages\parameterized\parameterized.py:620:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
..\AppData\Local\miniforge3\envs\mantidimaging-dev\lib\unittest\mock.py:1379: in patched
    return func(*newargs, **newkeywargs)
..\AppData\Local\miniforge3\envs\mantidimaging-dev\lib\site-packages\parameterized\parameterized.py:93: in dummy_func
    return orgfunc(*args, **kwargs)
..\AppData\Local\miniforge3\envs\mantidimaging-dev\lib\unittest\mock.py:1379: in patched
    return func(*newargs, **newkeywargs)
mantidimaging\gui\test\gui_system_loading_test.py:225: in test_load_with_start_stop_inc
    self.assertEqual(image_count, expected_count)
E   AssertionError: 1143 != 100
_____________________________________________ TestGuiSystemReconstruction.test_refine _____________________________________________

self = <mantidimaging.gui.test.gui_system_reconstruction_test.TestGuiSystemReconstruction testMethod=test_refine>

    def tearDown(self) -> None:
        wait_until(lambda: len(self.recon_window.presenter.async_tracker) == 0)
        self.recon_window.close()
        assert isinstance(self.main_window.recon, ReconstructWindowView)
        self.assertFalse(self.main_window.recon.isVisible())
        self._close_image_stacks()
>       self.mock_show_error_dialog.assert_not_called()

mantidimaging\gui\test\gui_system_reconstruction_test.py:42:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <MagicMock name='show_error_dialog' id='1915486344176'>

    def assert_not_called(self):
        """assert that the mock was never called.
        """
        if self.call_count != 0:
            msg = ("Expected '%s' to not have been called. Called %s times.%s"
                   % (self._mock_name or 'mock',
                      self.call_count,
                      self._calls_repr()))
>           raise AssertionError(msg)
E           AssertionError: Expected 'show_error_dialog' to not have been called. Called 1 times.
E           Calls: [call('Finding the COR failed.\n\n Error: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.')].

..\AppData\Local\miniforge3\envs\mantidimaging-dev\lib\unittest\mock.py:890: AssertionError
-------------------------------------------------------- Captured log call --------------------------------------------------------
ERROR    mantidimaging.gui.dialogs.async_task.task:task.py:53 Failed to execute task: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
Traceback (most recent call last):
  File "C:\Users\ddb29996\mantidimaging\mantidimaging\gui\dialogs\async_task\task.py", line 50, in run
    self.result = call_with_known_parameters(self.task_function, **self.kwargs)
  File "C:\Users\ddb29996\mantidimaging\mantidimaging\core\utility\func_call.py", line 12, in call_with_known_parameters
    return func(**ka)
  File "C:\Users\ddb29996\mantidimaging\mantidimaging\gui\windows\recon\model.py", line 249, in auto_find_minimisation_sqsum
    cor = reconstructor.find_cor(self.images, slice, initial_cor[idx], recon_params)
  File "C:\Users\ddb29996\mantidimaging\mantidimaging\core\reconstruct\astra_recon.py", line 102, in find_cor
    return minimize(minimizer_function, start_cor, method='nelder-mead', tol=0.1).x[0]
  File "C:\Users\ddb29996\AppData\Local\miniforge3\envs\mantidimaging-dev\lib\site-packages\scipy\optimize\_minimize.py", line 719, in minimize
    res = _minimize_neldermead(fun, x0, args, callback, bounds=bounds,
  File "C:\Users\ddb29996\AppData\Local\miniforge3\envs\mantidimaging-dev\lib\site-packages\scipy\optimize\_optimize.py", line 817, in _minimize_neldermead
    fsim[k] = func(sim[k])
  File "C:\Users\ddb29996\AppData\Local\miniforge3\envs\mantidimaging-dev\lib\site-packages\scipy\optimize\_optimize.py", line 526, in function_wrapper
    fx = function(np.copy(x), *(wrapper_args + args))
  File "C:\Users\ddb29996\mantidimaging\mantidimaging\core\reconstruct\astra_recon.py", line 100, in minimizer_function
    return -get_sumsq(AstraRecon.single_sino(images.sino(slice_idx), ScalarCoR(cor), proj_angles, recon_params))
  File "C:\Users\ddb29996\mantidimaging\mantidimaging\core\reconstruct\astra_recon.py", line 118, in single_sino
    vectors = vec_geom_init2d(proj_angles, 1.0, cor.to_vec(image_width).value)
  File "C:\Users\ddb29996\mantidimaging\mantidimaging\core\reconstruct\astra_recon.py", line 39, in vec_geom_init2d
    vectors[i, 2:4] = np.dot(rotation_matrix2d(theta), d0)[:]  # center of detector position
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
______________________________________________ TestGuiSystemWindows.test_loaded_data ______________________________________________

self = <mantidimaging.gui.test.gui_system_windows_test.TestGuiSystemWindows testMethod=test_loaded_data>

    def test_loaded_data(self):
        self._close_welcome()
        self._load_data_set()

        datasets = list(self.main_window.presenter.datasets)
        self.assertEqual(len(datasets), 1)
        dataset = datasets[0]
        self.assertIsNotNone(dataset.sample)
        self.assertIsNotNone(dataset.flat_before)
        self.assertIsNotNone(dataset.flat_after)
        self.assertIsNotNone(dataset.dark_before)
        self.assertIsNotNone(dataset.proj180deg)
        self.assertIsNone(dataset.dark_after)

>       self.assertTupleEqual(dataset.sample.data.shape, (100, 128, 128))
E       AssertionError: Tuples differ: (1143, 512, 512) != (100, 128, 128)
E
E       First differing element 0:
E       1143
E       100
E
E       - (1143, 512, 512)
E       + (100, 128, 128)

mantidimaging\gui\test\gui_system_windows_test.py:40: AssertionError
======================================================== warnings summary =========================================================
..\AppData\Local\miniforge3\envs\mantidimaging-dev\lib\site-packages\skimage\util\dtype.py:27
  C:\Users\ddb29996\AppData\Local\miniforge3\envs\mantidimaging-dev\lib\site-packages\skimage\util\dtype.py:27: DeprecationWarning: `np.bool8` is a deprecated alias for `np.bool_`.  (Deprecated NumPy 1.24)
    np.bool8: (False, True),

mantidimaging/gui/test/gui_system_spectrum_test.py::TestGuiSpectrumViewer::test_add_roi
mantidimaging/gui/test/gui_system_spectrum_test.py::TestGuiSpectrumViewer::test_add_roi
mantidimaging/gui/test/gui_system_spectrum_test.py::TestGuiSpectrumViewer::test_spectrum_window_opens_with_data_in_default_state
mantidimaging/gui/test/gui_system_spectrum_test.py::TestGuiSpectrumViewer::test_spectrum_window_opens_with_data_in_default_state
  C:\Users\ddb29996\AppData\Local\miniforge3\envs\mantidimaging-dev\lib\site-packages\pyqtgraph\graphicsItems\PlotItem\PlotItem.py:509: UserWarning: Item already added to PlotItem, ignoring.
    warnings.warn('Item already added to PlotItem, ignoring.')

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ 3 failed, 42 passed, 2 xfailed, 5 warnings in 1219.22s (0:20:19) =================================

[samtygier-stfc]

Looks good, Working here.