Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

open all datasets at zonal entry point #832

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Conversation

asinghvi17
Copy link
Collaborator

this should massively increase performance for lazy=true rasters. But it needs tests with rasters of every form.

this should massively increase performance for `lazy=true` rasters.  But it needs tests with rasters of every form.
@rafaqz
Copy link
Owner

rafaqz commented Dec 9, 2024

Yeah probably a lazy test is needed, there may be one already, it can go with GDAL zonal tests

@rafaqz
Copy link
Owner

rafaqz commented Jan 12, 2025

We really need chunked burning for this to have much of an effect.

@asinghvi17
Copy link
Collaborator Author

asinghvi17 commented Jan 13, 2025

We really need chunked burning for this to have much of an effect.

Doesn't that already work with the exception of #816? Then the chunking only has to be done on the top level...similar to what RangeExtractor does

Also, the main use is to avoid the performance footgun IMO, not necessarily for that. People will get caught in the trap of not using open very easily...

@rafaqz
Copy link
Owner

rafaqz commented Jan 13, 2025

I'm pretty sure #814 is fixed now.

But there will be other memory problems from not chunking, although I guess they are with rasterize proper as those paths aren't really hit in zonal.

Ok I'll just merge this if there is a lazy test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants