DensMAP support

[keller-mark]

I tried to follow the feedback described in a previous PR that contributed DensMAP #2684 (comment) but re-implemented on top of the state of the current scanpy main branch.

I did not add release notes because the contributor guide says to wait for PR feedback https://scanpy.readthedocs.io/en/latest/dev/documentation.html#adding-to-the-docs

• Closes densmap #1619
• Closes DensMAP support #2684
• Tests included or not required because:

• Release notes not necessary because:

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 75.46%. Comparing base (05ab694) to head (3b68fe2).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2946      +/-   ##
==========================================
+ Coverage   75.43%   75.46%   +0.02%     
==========================================
  Files         112      113       +1     
  Lines       13235    13247      +12     
==========================================
+ Hits         9984     9997      +13     
+ Misses       3251     3250       -1     

Files with missing lines	Coverage Δ
src/scanpy/tools/__init__.py	91.30% <ø> (ø)
src/scanpy/tools/_types.py	100.00% <100.00%> (ø)
src/scanpy/tools/_umap.py	73.52% <100.00%> (+4.17%)	⬆️

[keller-mark]

Hi, it seems the Check if milestone or "no milestone" label is present GH Actions check fails but I am not able to add a PR label

[flying-sheep]

Hi, thanks for you contribution! Please add a plot test that shows sufficiently different results from umap.

[flying-sheep]

I’d rather not maintain that. Just method='densmap' should be sufficient, but if densmap = partial(umap, method='densmap') works and displays in the docs as expected, I’d consider it.

If you want to make it more visible, you could add an example to the sc.pl.umap docs and maybe even a tutorial about what it’s for and how to best use it

[keller-mark]

I tried the partial option, but there seem to be issues with inference of the types from the parent umap function when generating the autodoc for sphinx that are not straightforward to resolve.

[keller-mark]

I have added a plotting test - this is how the UMAP and DensMAP results for this test look

[keller-mark]

Hi, it seems that the plotting test that I added is failing in Azure CI, despite passing locally. I have read the plotting test notes at https://scanpy.readthedocs.io/en/stable/dev/testing.html#plotting-tests and https://github.com/scverse/scanpy/blob/main/docs/dev/ci.md#plotting-tests but cannot seem to find a place to view the generated image files in the Azure CI Attachments interface.

https://dev.azure.com/scverse/scanpy/_build/results?buildId=9158&view=ms.vss-test-web.build-test-results-tab&runId=30098&resultId=100650&paneView=attachments

E       AssertionError: Image files did not match.
E       RMS Value:  16.984969731422655
E       Expected:   file:///home/vsts/work/1/s/tests/_images/densmap_nocolor/expected.png
E       Actual:     file:///home/vsts/work/1/s/tests/_images/densmap_nocolor/actual.png
E       Difference: file:///home/vsts/work/1/s/tests/_images/densmap_nocolor/actual-failed-diff.png
E       Tolerance:  15

[keller-mark]

I could use some help debugging this, the image-diffing test keeps failing with RMS values slightly above 15 but only in the PyTest minimum_versions environment.

[ilan-gold]

@keller-mark I'm not super familiar with the azure but I know you can manually upload an artifact. Also, have you tried locally? In the hatch.toml you can find how to create a min environment on your machine.

[ilan-gold]

Relatedly, I wonder if there is something with you machine vs. remote. You can output the densmap results as a zarr zip store and upload that as an artifact as well.

[keller-mark]

Thanks, I installed the min-versions dependencies locally in a python 3.10 environment and was able to reproduce the issue.

The same random_state parameter value (I tried both zero and non-zero values) is producing different results between the min-versions environment and all others. I tried random_state=np.random.RandomState(1) based on the numpy legacy random docs in case it was due to numpy v1 vs v2 but it does not resolve the issue.

Please add a plot test that shows sufficiently different results from umap.

Since ultimately the plotting is just using the standard sc.pl.embedding, is a visual test really necessary here? That code path is already extensively tested. Instead I could add a numerical (as opposed to visual) test that checks that the method="densmap" produces sufficiently different results than method="umap"

[keller-mark]

This test also fails, confirming the issue is not specific to DensMAP: 8f8787a