Releases: constantinpape/z5
Fix regression when reading data
Also switch to github actions for CI.
Fix issues with reading n5 files written with zarr-python
2.0.7 Bump version
Fix issues with recent std::filesystem versions
2.0.6 Bump version
Add get_chunk_shape to z5py.Dataset
Get the actual chunk shape from the z5py.Dataset API
n5 blosc compression and more bugfixes
-
Implement blosc compression for n5
-
Add factory functions for filesystem handles that accept c strings
-
Bugfixes:
-- Fix default blosc compression value for C++ (thanks to @gdkrmr)
-- Add default compression options for xz and lz5 in C++
-- Acceptpathlib.Path
inz5py.File
constructor (thanks to @k-dominik)
-- Fix appveyor for PRs (thanks to @k-dominik)
Bugfixes and API improvement
Fixed bugs:
- Fix egde cases when reading varlen chunks containing NaN
- Pass fillvalue in python dataset constructor
- Enable NaN, Inf and -Inf fillvalues (thanks to @gdkrmr)
- Fix issue with no compression in zarr (thanks to @gdkrmr)
API Improvements:
- Implement
parent
attribute forFile
,Group
andDataset
(same ash5py
API) - Implement
name
attribute forFile
,Group
andDataset
(same ash5py
API) - Implement
filename
attribute forFile
(same ash5py
API)
Minor update to C++ signature
Bugfix:
Update the function signature of createDataset
to fix unresolvable overload.
For DOI
Create release for a zenodo DOI.
C++ API overhaul for cloud support
This release overhauls the C++ API in order to support multiple storage backends and mirror the python API closer.
The new API is based on handles for the different storage implementations.
For example, the handle for a root file on the filesystem is z5::filesystem::handle::File
, the handle for a root file in an s3 bucket is z5::s3::handle::File
.
The handles can be passed to factory functions in order to create or open the corresponding objects or to the attribute functions in order to read or write metadata attributes.
Datasets have an additional dataset object that needs to inherit from the Dataset class.
Its API defines access to the data and it can be obtained by the factory functions z5::openDataset
/ z5::createDataset
.
For example, creating a root n5-file on the filesystem and then creating a group and a dataset in it work like this:
// filesystem handle for the root file
z5::filesystem::handle::File f("/path/to/file.n5");
// create the file in n5 format
const bool createAsZarr = false;
z5::createFile(f, createAsZarr);
// create a new group in the root file and get its handle
createGroup(f, "newGroup");
z5::filesystem::handle::Group g(f, "newGroup");
// create a dataset in the root file
// (we could also use the group handle 'g' to create a dataset in the group the same way
std::vector<std::size_t> shape = {100, 100, 100};
std::vector<std::size_t> chunks = {10, 10, 10};
auto ds = z5::createDataset(f, "data", "float32", shape, chunks);
// get the dataset handle, which can be e.g. used to write attributes
z5::filesystem::handle::Dataset dsHandle(f, "data");
Note that this release contains an implementation of the AWS-S3 backend, which is not fully functional yet.
Any help in finishing this implementation or contributing other cloud based backends is highly appreciated.
Unfortunately, the changes introduced a few build issues, any help in fixing this would be highly appreciated.
See about mac build issues on conda-forge.
Drop py2 support and optional boost filesystem
- Drop support for python 2 and python 3.5
- Make boost filesystem dependency optional (still used as fallback if C++17 not supported)