Skip to content

Commit

Permalink
Use mountpoint-s3-client-0.8.1 and mountpoint-s3-crt-0.7.0. (awslabs#202
Browse files Browse the repository at this point in the history
)

Update version number to 1.2.3.
  • Loading branch information
dnanuti authored Apr 11, 2024
1 parent 23806df commit b4224d1
Show file tree
Hide file tree
Showing 5 changed files with 29 additions and 23 deletions.
26 changes: 16 additions & 10 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,27 @@
## Unreleased
## v1.2.3 (April 11, 2024)

### New features
* Update S3ClientConfig to pass in the configuration for allowing unsigned requests, under boolean flag `unsigned`.
* Improve the performance of `s3reader` when utilized with `pytorch.load` by incorporating support for the `readinto` method.
* Add support for passing an optional custom endpoint to S3LightningCheckpoint constructor method.
* Update `S3ClientConfig` to pass in the configuration for allowing unsigned requests, under boolean flag `unsigned`.
* Improve the performance of `S3Reader` when utilized with `pytorch.load` by incorporating support for the `readinto` method.
* **[Experimental]** Add support for passing an optional custom endpoint to `S3LightningCheckpoint` constructor method.

### Breaking changes
* No breaking changes.

## v1.2.2 (March 22, 2024)

### New features
* Expose a new class, S3ClientConfig, with `throughput_target_gbps` and `part_size` parameters of the inner S3 client.
* Expose a new class, `S3ClientConfig`, with `throughput_target_gbps` and `part_size` parameters of the inner S3 client.

### Breaking changes
* No breaking changes.

## v1.2.1 (March 14, 2024)

### Breaking changes
* Separate completely Rust logs and Python logs. Logs from Rust components used for debugging purposes
are configured through the following environment variables: S3_TORCH_CONNECTOR_DEBUG_LOGS,
S3_TORCH_CONNECTOR_LOGS_DIR_PATH.
are configured through the following environment variables: `S3_TORCH_CONNECTOR_DEBUG_LOGS`,
`S3_TORCH_CONNECTOR_LOGS_DIR_PATH`.

## v1.2.0 (March 13, 2024)

Expand Down Expand Up @@ -72,7 +77,8 @@ S3_TORCH_CONNECTOR_LOGS_DIR_PATH.

### New features
* Update crates and Mountpoint dependencies.
* Avoid excessive memory consumption when utilizing s3map_dataset. Issue [#89](https://github.com/awslabs/s3-connector-for-pytorch/issues/89).
* Avoid excessive memory consumption when utilizing `S3MapDataset`.
Issue [#89](https://github.com/awslabs/s3-connector-for-pytorch/issues/89).
* Run all tests against S3 and S3 Express.

### Breaking changes
Expand All @@ -92,10 +98,10 @@ S3_TORCH_CONNECTOR_LOGS_DIR_PATH.
* The Amazon S3 Connector for PyTorch delivers high throughput for PyTorch training jobs that access and store data in Amazon S3.

### New features
* S3IterableDataset and S3MapDataset, which allow building either an iterable-style or map-style dataset, using your S3
* `S3IterableDataset` and `S3MapDataset`, which allow building either an iterable-style or map-style dataset, using your S3
stored data, by specifying an S3 URI (a bucket and optional prefix) and the region the bucket is in.
* Support for multiprocess data loading for the above datasets.
* S3Checkpoint, an interface for saving and loading model checkpoints directly to and from an S3 bucket.
* `S3Checkpoint`, an interface for saving and loading model checkpoints directly to and from an S3 bucket.

### Breaking changes
* No breaking changes.
4 changes: 2 additions & 2 deletions s3torchconnector/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "s3torchconnector"
version = "1.2.2"
version = "1.2.3"
description = "S3 connector integration for PyTorch"
requires-python = ">=3.8,<3.13"
readme = "README.md"
Expand All @@ -23,7 +23,7 @@ classifiers = [

dependencies = [
"torch >= 2.0.1",
"s3torchconnectorclient >= 1.2.2",
"s3torchconnectorclient >= 1.2.3",
]

[project.optional-dependencies]
Expand Down
14 changes: 7 additions & 7 deletions s3torchconnectorclient/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions s3torchconnectorclient/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "s3torchconnectorclient"
version = "1.2.2"
version = "1.2.3"
edition = "2021"
publish = false
license = "BSD-3-Clause"
Expand All @@ -19,8 +19,8 @@ built = "0.7"
pyo3 = { version = "0.19.2" }
pyo3-log = "0.8.3"
futures = "0.3.28"
mountpoint-s3-client = { version = "0.8.0", features = ["mock"] }
mountpoint-s3-crt = "0.6.2"
mountpoint-s3-client = { version = "0.8.1", features = ["mock"] }
mountpoint-s3-crt = "0.7.0"
log = "0.4.20"
tracing = { version = "0.1.40", default-features = false, features = ["std", "log"] }
tracing-subscriber = { version = "0.3.18", features = ["fmt", "env-filter"]}
Expand Down
2 changes: 1 addition & 1 deletion s3torchconnectorclient/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "s3torchconnectorclient"
version = "1.2.2"
version = "1.2.3"
description = "Internal S3 client implementation for s3torchconnector"
requires-python = ">=3.8,<3.13"
readme = "README.md"
Expand Down

0 comments on commit b4224d1

Please sign in to comment.