Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

upgrade ndarray to 0.16 #569

Merged
merged 1 commit into from
Oct 16, 2024
Merged

Conversation

nmandery
Copy link
Contributor

@nmandery nmandery commented Oct 6, 2024

This PR upgrades the ndarray dependency from 0.15 to 0.16

  • I agree to follow the project's code of conduct.
  • I added an entry to CHANGES.md if knowledge of this change could be valuable to users.

@lnicola
Copy link
Member

lnicola commented Oct 10, 2024

Thanks, but I don't think this is right (just as the previous code wasn't right). For reference, I saw this example in rust-ndarray/ndarray#994:

let a = Array2::<f64>::zeros([3, 4]).slice_move(s![1.., ..]);
let b = Array2::<f64>::zeros([2, 4]);
assert!(a.is_standard_layout());
assert!(b.is_standard_layout());
assert_eq!(a.shape(), b.shape());
assert_eq!(a.strides(), b.strides());
assert_ne!(
    unsafe { a.as_ptr().offset_from(a.into_raw_vec().as_ptr()) }, // 4
    unsafe { b.as_ptr().offset_from(b.into_raw_vec().as_ptr()) }, // 0
);

Which implies that slicing sometimes avoids copying the data by bumping an offset from the start of the storage (the example strips off the first row).

So the offset is not always 0 (of course, otherwise it wouldn't be there), and we should handle it, either by copying the data to a new Vec, or by introducing a similar offset concept, which would probably be quite confusing to use.

@nmandery
Copy link
Contributor Author

I understood having a standard layout means also having an offset of 0, but I am afraid you are right. So we need to copy the data (like it is currently done for the non-standard-layout) in case of an offset.

I will add this to the PR later today.

@lnicola
Copy link
Member

lnicola commented Oct 10, 2024

I'm again wondering if we should fully switch to ndarray (see #570).

if let Some(offset) = offset {
data.into_iter().skip(offset).collect()
} else {
data
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's also handle offset == 0 the same way.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done - I removed the if

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That might do an extra copy in the zero offset case.

value.into_raw_vec()
let (data, offset) = value.into_raw_vec_and_offset();
if let Some(offset) = offset {
data.into_iter().skip(offset).collect()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
data.into_iter().skip(offset).collect()
data[offset].to_vec()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yepp, done

@nmandery nmandery force-pushed the upgrade-ndarray-to-0.16 branch 2 times, most recently from 229daaf to d1fa058 Compare October 10, 2024 19:09
@lnicola lnicola self-assigned this Oct 15, 2024
@lnicola lnicola force-pushed the upgrade-ndarray-to-0.16 branch from d1fa058 to ed28a25 Compare October 16, 2024 17:39
@lnicola lnicola merged commit 5a500bd into georust:master Oct 16, 2024
15 checks passed
@lnicola
Copy link
Member

lnicola commented Oct 17, 2024

@rouault looks like I've just won the lottery in https://github.com/georust/gdal/actions/runs/11372075639/job/31635584318:

OGR2SQLITE_Setup() failed due to sqlite3_api == nullptr

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants