Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Build against TF2.16 #2866

Closed
wants to merge 11 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 21 additions & 27 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ permissions:

env:
MIN_PY_VERSION: '3.9'
MAX_PY_VERSION: '3.11'
MAX_PY_VERSION: '3.12'

jobs:
test-with-bazel:
Expand All @@ -46,22 +46,19 @@ jobs:
os: ['macos-12', 'ubuntu-20.04']
# Removing windows builds due to broken upstream package https://github.com/tensorflow/tensorflow/issues/61830
# os: ['macos-12', 'windows-2019', 'ubuntu-20.04']
py-version: ['3.9', '3.10', '3.11']
tf-version: ['2.13.1', '2.14.0', '2.15.0']
py-version: ['3.9', '3.10', '3.11', '3.12']
tf-version: ['2.14.1', '2.15.1', '2.16.1']
cpu: ['x86']
include:
- os: 'macos-12'
cpu: 'arm64'
tf-version: '2.15.0'
py-version: '3.9'
- os: 'macos-12'
cpu: 'arm64'
tf-version: '2.15.0'
py-version: '3.10'
- os: 'macos-12'
cpu: 'arm64'
tf-version: '2.15.0'
py-version: '3.11'
py-version: 3.12
tf-version: '2.16.1'
exclude:
- py-version: '3.12'
tf-version: '2.14.1'
- py-version: '3.12'
tf-version: '2.15.1'
fail-fast: false
steps:
- uses: actions/[email protected]
Expand All @@ -71,7 +68,7 @@ jobs:
script: |
const commit_details = await github.git.getCommit({owner: context.repo.owner, repo: context.repo.repo, commit_sha: context.sha});
return commit_details.data.author.date
- if: matrix.tf-version != '2.15.0'
- if: matrix.tf-version != '2.16.1'
shell: bash
run: echo "SKIP_CUSTOM_OP_TESTS=--skip-custom-ops" >> $GITHUB_ENV
- if: github.event_name == 'push'
Expand Down Expand Up @@ -107,22 +104,19 @@ jobs:
os: ['macOS', 'Linux']
# Removing windows builds due to broken upstream package https://github.com/tensorflow/tensorflow/issues/61830
# os: ['macOS', 'Windows', 'Linux']
py-version: ['3.9', '3.10', '3.11']
tf-version: ['2.15.0']
py-version: ['3.9', '3.10', '3.11', '3.12']
tf-version: ['2.16.1']
cpu: ['x86']
include:
- os: 'macOS'
cpu: 'arm64'
tf-version: '2.15.0'
py-version: '3.9'
- os: 'macOS'
cpu: 'arm64'
tf-version: '2.15.0'
py-version: '3.10'
- os: 'macOS'
- os: 'macos-12'
cpu: 'arm64'
tf-version: '2.15.0'
py-version: '3.11'
py-version: 3.12
tf-version: '2.16.1'
exclude:
- py-version: '3.12'
tf-version: '2.14.1'
- py-version: '3.12'
tf-version: '2.15.1'
fail-fast: false
if: (github.event_name == 'push' && github.ref == 'refs/heads/master') || github.event_name == 'release'
steps:
Expand Down
10 changes: 5 additions & 5 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,8 +173,8 @@ conda activate my_dev_environement
Just run from the root:

```bash
pip install tensorflow==2.15.0
# you can use "pip install tensorflow-cpu==2.15.0" too if you're not testing on gpu.
pip install tensorflow==2.16.1
# you can use "pip install tensorflow-cpu==2.16.1" too if you're not testing on gpu.
pip install -e ./
```

Expand Down Expand Up @@ -262,7 +262,7 @@ If you need a custom C++/Cuda op for your test, compile your ops with

```bash
python configure.py
pip install tensorflow==2.15.0 -e ./ -r tools/install_deps/pytest.txt
pip install tensorflow==2.16.1 -e ./ -r tools/install_deps/pytest.txt
bash tools/install_so_files.sh # Linux/macos/WSL2
sh tools/install_so_files.sh # PowerShell
```
Expand Down Expand Up @@ -290,7 +290,7 @@ docker run --gpus all --rm -it -v ${PWD}:/addons -w /addons gcr.io/tensorflow-te

Configure:
```bash
python3 -m pip install tensorflow==2.15.0
python3 -m pip install tensorflow==2.16.1
python3 ./configure.py # Links project with TensorFlow dependency
```

Expand Down Expand Up @@ -329,7 +329,7 @@ quickly, as Bazel has great support for caching and distributed testing.
To test with Bazel:

```bash
python3 -m pip install tensorflow==2.15.0
python3 -m pip install tensorflow==2.16.1
python3 configure.py
python3 -m pip install -r tools/install_deps/pytest.txt
bazel test -c opt -k \
Expand Down
35 changes: 30 additions & 5 deletions WORKSPACE
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,41 @@ tf_configure(

http_archive(
name = "org_tensorflow",
patches = [
"//build_deps/tf_dependency:tf.patch",
patch_cmds = [
"""sed -i.bak 's/cython-3.0.3/cython-3.0.0a11/g' tensorflow/workspace2.bzl""",
"""sed -i.bak 's/3.0.3.tar.gz/3.0.0a11.tar.gz/g' tensorflow/workspace2.bzl""",
"""sed -i.bak 's/0c2eae8a4ceab7955be1e11a4ddc5dcc3aa06ce22ad594262f1555b9d10667f0/08dbdb6aa003f03e65879de8f899f87c8c718cd874a31ae9c29f8726da2f5ab0/g' tensorflow/workspace2.bzl""",
],
sha256 = "9cec5acb0ecf2d47b16891f8bc5bc6fbfdffe1700bdadc0d9ebe27ea34f0c220",
strip_prefix = "tensorflow-2.15.0",
sha256 = "c729e56efc945c6df08efe5c9f5b8b89329c7c91b8f40ad2bb3e13900bd4876d",
strip_prefix = "tensorflow-2.16.1",
urls = [
"https://github.com/tensorflow/tensorflow/archive/refs/tags/v2.15.0.tar.gz",
"https://github.com/tensorflow/tensorflow/archive/refs/tags/v2.16.1.tar.gz",
],
)

http_archive(
name = "rules_python",
sha256 = "84aec9e21cc56fbc7f1335035a71c850d1b9b5cc6ff497306f84cced9a769841",
strip_prefix = "rules_python-0.23.1",
url = "https://github.com/bazelbuild/rules_python/releases/download/0.23.1/rules_python-0.23.1.tar.gz",
)

load("@rules_python//python:repositories.bzl", "python_register_toolchains")
load(
"@org_tensorflow//tensorflow/tools/toolchains/python:python_repo.bzl",
"python_repository",
)

python_repository(name = "python_version_repo")

load("@python_version_repo//:py_version.bzl", "HERMETIC_PYTHON_VERSION")

python_register_toolchains(
name = "python",
ignore_root_user_error = True,
python_version = HERMETIC_PYTHON_VERSION,
)

load("@org_tensorflow//tensorflow:workspace3.bzl", "tf_workspace3")

tf_workspace3()
Expand Down
4 changes: 2 additions & 2 deletions tensorflow_addons/utils/resource_loader.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@

import tensorflow as tf

INCLUSIVE_MIN_TF_VERSION_FOR_ABI_COMPATIBILITY = "2.15.0"
EXCLUSIVE_MAX_TF_VERSION_FOR_ABI_COMPATIBILITY = "2.16.0"
INCLUSIVE_MIN_TF_VERSION_FOR_ABI_COMPATIBILITY = "2.16.0"
EXCLUSIVE_MAX_TF_VERSION_FOR_ABI_COMPATIBILITY = "2.17.0"
abi_warning_already_raised = False
SKIP_CUSTOM_OPS = False

Expand Down
4 changes: 2 additions & 2 deletions tensorflow_addons/version.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@
"""Define TensorFlow Addons version information."""

# Required TensorFlow version [min, max)
INCLUSIVE_MIN_TF_VERSION = "2.13.0"
EXCLUSIVE_MAX_TF_VERSION = "2.16.0"
INCLUSIVE_MIN_TF_VERSION = "2.14.0"
EXCLUSIVE_MAX_TF_VERSION = "2.17.0"

# We follow Semantic Versioning (https://semver.org/)
_MAJOR_VERSION = "0"
Expand Down
2 changes: 1 addition & 1 deletion tools/build_dev_container.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ set -x -e

docker build \
-f tools/docker/dev_container.Dockerfile \
--build-arg TF_VERSION=2.15.0 \
--build-arg TF_VERSION=2.16.1 \
--build-arg TF_PACKAGE=tensorflow \
--build-arg PY_VERSION=$PY_VERSION \
--no-cache \
Expand Down
4 changes: 2 additions & 2 deletions tools/docker/build_wheel.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
#syntax=docker/dockerfile:1.1.5-experimental
ARG PY_VERSION
FROM tensorflow/build:2.15-python$PY_VERSION as base_install
FROM tensorflow/build:2.16-python$PY_VERSION as base_install

ENV TF_NEED_CUDA="1"
ARG PY_VERSION
ARG TF_VERSION

# TODO: Temporary due to build bug https://github.com/pypa/pip/issues/11770
RUN python -m pip install pip==22.3.1
RUN python -m pip install --upgrade setuptools

# TODO: Remove this if tensorflow/build container removes their keras-nightly install
# https://github.com/tensorflow/build/issues/78
Expand Down
2 changes: 1 addition & 1 deletion tools/docker/cpu_tests.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#syntax=docker/dockerfile:1.1.5-experimental
FROM python:3.9 as build_wheel

ARG TF_VERSION=2.15.0
ARG TF_VERSION=2.16.1
RUN pip install --default-timeout=1000 tensorflow-cpu==$TF_VERSION

RUN apt-get update && apt-get install -y sudo rsync
Expand Down
2 changes: 1 addition & 1 deletion tools/install_deps/tensorflow-cpu.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
tensorflow-cpu~=2.15.0
tensorflow-cpu~=2.16.1
2 changes: 1 addition & 1 deletion tools/install_deps/tensorflow.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
tensorflow~=2.15.0
tensorflow~=2.16.1
2 changes: 1 addition & 1 deletion tools/run_gpu_tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ export DOCKER_BUILDKIT=1
docker build \
-f tools/docker/build_wheel.Dockerfile \
--target tfa_gpu_tests \
--build-arg TF_VERSION=2.15.0 \
--build-arg TF_VERSION=2.16.1 \
--build-arg PY_VERSION=3.9 \
-t tfa_gpu_tests ./
docker run --rm -t --gpus=all --shm-size=512m tfa_gpu_tests
Loading