Difference in test results in Windows and Unix

[astrofle]

During a merge it was noticed that the tests being ran on windows failed, while the same tests passed on Ubuntu and MacOS.
This is the line causing problems.
This is the failed action.

We should understand why this happens moving forward.

[tchamberlin]

This should be resolved by #120. I'm not exactly sure why there's a mismatch between OSes, but the core issue I think is that we had project.requires-python set to >=3.9, and tool.hatch.envs.test.matrix set to include 38. So it tried to test against a version that wasn't available

But, #120 synchronizes all of these version sets, so this should be fine now. And given that #120 passed all checks, I'm going to close this

[astrofle]

Marc modified the test that was failing. The difference is not large, but I'm still curious. I'll reopen until we have a better answer.

[tchamberlin]

Ah, okay, I see the change, which was merged as part of PR #117

I suspect that this is a precision difference between Windows and other platforms -- the arrays under test are float128, which is perhaps not available on the GitHub Windows test runner

[tchamberlin]

Okay, I have verified that there is a precision difference between the Windows and Linux/Mac test runners.

In TestPSScan.test_blank_integrations, ta1[0].flux.value is float128 on Linux, but float64 on Windows. This means that on Linux, ps_sb[0].calibrated(0).flux.value (which is float64 on all platforms) is being cast to float128 prior to the comparison, but on Windows it is not.

I'm not sure if 128-bit precision is actually necessary or not

Output from windows-latest, 3.10:

test_blank_integrations: cal_flux_value.dtype=dtype('float64')
test_blank_integrations: cal_flux_value=array([-0.27595341, -0.68388849, -0.10646429, ...,  0.40062377,
       -0.19491617, -0.33765927])
test_blank_integrations: raw_flux_value.dtype=dtype('float64')
test_blank_integrations: (cal_flux_value - raw_flux_value).dtype=dtype('float64')
test_blank_integrations: cal_flux_value - raw_flux_value=array([0.00000000e+00, 0.00000000e+00, 1.38777878e-17, ...,
       0.00000000e+00, 0.00000000e+00, 0.00000000e+00], dtype=float64)

Output from ubuntu-latest, 3.10

test_blank_integrations: cal_flux_value.dtype=dtype('float64')
test_blank_integrations: cal_flux_value=array([-0.27595341, -0.68388849, -0.10646429, ...,  0.40062377,
       -0.19491617, -0.33765927])
test_blank_integrations: raw_flux_value.dtype=dtype('float128')
test_blank_integrations: (cal_flux_value - raw_flux_value).dtype=dtype('float128')
test_blank_integrations: cal_flux_value - raw_flux_value=array([ 0.00000000e+00,  0.00000000e+00,  6.77626358e-21, ...,
       -2.71050543e-20,  0.00000000e+00,  0.00000000e+00], dtype=float128)

Full output is available here

[astrofle]

Now that we are not using float128 we can re run the action with the previous criteria and see if it still fails under Windows.