Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ICU-22606 Create an (almost) full release from CI #3222

Merged
merged 5 commits into from
Sep 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .github/Dockerfile_fedora
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
FROM fedora:latest

RUN dnf install -y gcc-c++ zip unzip git-core git-lfs doxygen
RUN git lfs install --skip-repo \
&& ln -s /usr/bin/python3 /usr/bin/python

WORKDIR /root

ENTRYPOINT [ "/bin/bash" ]
38 changes: 38 additions & 0 deletions .github/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
## How to create a Fedora docker image

For the general process and concepts see:
https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry

For our case I replaced the generic names with our own owner / repo / names / etc.

Run
```
docker login ghcr.io
```

When prompted use these:

* **User:** the github user
* **Password:** the github token

Update the timestamp (`20240929`) with the current date, ISO style:
```
docker build --tag ghcr.io/unicode-org/fedora-docker-gcr:20240929 -f Dockerfile_fedora .
docker push ghcr.io/unicode-org/fedora-docker-gcr:20240929
```

For more info see:
https://docs.github.com/en/actions/use-cases-and-examples/publishing-packages/publishing-docker-images

and:
https://stackoverflow.com/questions/64033686/how-can-i-use-private-docker-image-in-github-actions

To consider: generate and publish the docker image from a GitHub action.

---

The `DOCKER_CONTAINER_USER_NAME` and `DOCKER_CONTAINER_REGISTRY_TOKEN` used
in the action file for user and password are secrets already created.

They can be any GitHub user + token with the proper access rights.
Right now this is a token of the icu-robot account.
42 changes: 31 additions & 11 deletions .github/workflows/icu4c.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,10 @@ on:
workflow_dispatch:
# To trigger the Env Test workflow manually, follow the instructions in
# https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow
inputs:
gitReleaseTag:
description: 'Release tag to upload to. Must start with "release-"'
type: string

# For non-release branches (namely: PRs), only run CI on the most recent commit. Cancel
# runs on previous commits mid-flight when new commits are pushed.
Expand All @@ -48,7 +52,7 @@ jobs:
cd icu4c/source;
./runConfigureICU Linux --disable-renaming;
# Fail if 'warning:' appears in doxygen's output, but ignore warnings from file Doxyfile.
# Regex note: (?! ... ) is a negative lookahead. Succeed if the pattern is not present.
# Regex note: (?! ... ) is a negative lookahead. Succeed if the pattern is not present.
set +o pipefail && make doc 2>&1 | tee doxygen.log && ( ! grep -P 'warning:(?! .* file .?Doxyfile)' doxygen.log )

# gcc debug build.
Expand Down Expand Up @@ -316,13 +320,13 @@ jobs:
run: |
cd icu4c/source && ./runConfigureICU --enable-debug --disable-release Linux/clang && make -j -l4.5 check

# Clang Linux 18 with CPP20 and treat warnings as errors
# Clang Linux 18 with CPP20 and treat warnings as errors
clang18-cpp20-warning-as-errors:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
flags:
flags:
- '-std=c++20'
- '-std=c++20 -stdlib=libc++'
steps:
Expand Down Expand Up @@ -360,7 +364,7 @@ jobs:
strategy:
fail-fast: false
matrix:
include:
include:
- test_flags: 'x64 Debug'
build_flags: '/p:Configuration=Debug /p:Platform=x64'
- test_flags: 'x86 Debug'
Expand Down Expand Up @@ -405,11 +409,13 @@ jobs:
# Windows MSVC distribution release
windows-msvc-dist-release:
runs-on: windows-latest
permissions:
contents: write # So that we can upload to release
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
include:
include:
- arch: 'x64'
plat: 'x64'
win_ver: 'Win64'
Expand Down Expand Up @@ -465,6 +471,12 @@ jobs:
with:
name: icu4c.${{ matrix.win_ver }}.run_#${{ github.run_number }}
path: icu4c/source/dist/${{ env.newZipName }}.zip
- name: Upload to release
if: ${{ inputs.gitReleaseTag && startsWith(inputs.gitReleaseTag, 'release-') }}
run: |
gh release upload ${{ inputs.gitReleaseTag }} icu4c/source/dist/${{ env.newZipName }}.zip --clobber
env:
GH_TOKEN: ${{ github.token }}

# Window MSYS2 tests
windows-msys2-gcc-x86_64:
Expand All @@ -481,9 +493,9 @@ jobs:
with:
update: true
msystem: mingw64
install: >
base-devel
mingw-w64-x86_64-toolchain
install: >
base-devel
mingw-w64-x86_64-toolchain
make
- name: 'Verify MinGW Installation'
run: gcc --version
Expand Down Expand Up @@ -824,7 +836,7 @@ jobs:
run: |
cd icu4c/source
make -j -l4.5 check

# https://unicode-org.github.io/icu/processes/release/tasks/healthy-code.html#test-uconfig_no_conversion
icu4c-uconfig-no-conversion:
runs-on: ubuntu-latest
Expand All @@ -848,6 +860,8 @@ jobs:
# Workflow for ICU Export Data for ICU4X
icu4c-icuexportdata:
runs-on: ubuntu-latest
permissions:
contents: write # So that we can upload to release
timeout-minutes: 30
steps:
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
Expand Down Expand Up @@ -910,9 +924,15 @@ jobs:
- name: Zip
run: |
cd icu4c/source/icuexportdata
zip -r ../../../icuexportdata_tag-goes-here.zip .
zip -r ../../../icuexportdata_${{ inputs.gitReleaseTag }}.zip .
echeran marked this conversation as resolved.
Show resolved Hide resolved
- name: Publish Artifact
uses: actions/upload-artifact@v4
with:
name: icuexportdata_output
path: icuexportdata_tag-goes-here.zip
path: icuexportdata_*.zip
- name: Upload to release
if: ${{ inputs.gitReleaseTag && startsWith(inputs.gitReleaseTag, 'release-') }}
run: |
gh release upload ${{ inputs.gitReleaseTag }} icuexportdata_*.zip --clobber
env:
GH_TOKEN: ${{ github.token }}
69 changes: 69 additions & 0 deletions .github/workflows/release-check-sign.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
name: Release - Create checksums and GPG sign

on:
workflow_dispatch:
inputs:
gitReleaseTag:
description: 'Release tag to upload to. Must start with "release-"'
type: string

env:
RELEASE_FOLDER: '${{ github.workspace }}/releaseDist'

jobs:
sign_and_checksums:
if: ${{ inputs.gitReleaseTag && startsWith(inputs.gitReleaseTag, 'release-') }}
runs-on: ubuntu-latest
environment: release-env

permissions:
contents: write # So that we can upload to release

steps:

- name: Checkout and setup
uses: actions/checkout@v4
with:
lfs: true

- name: Set up JDK
uses: actions/setup-java@v4
with:
java-version: '11'
distribution: 'temurin'
gpg-private-key: ${{ secrets.MAVEN_GPG_PRIVATE_KEY }}
gpg-passphrase: MAVEN_GPG_PASSPHRASE

- name: Get all release files
run: |
mkdir -p ${RELEASE_FOLDER}
pushd ${RELEASE_FOLDER}
gh release download ${{ inputs.gitReleaseTag }} -p "*.zip" -p "*.tgz" -p "*.jar" --repo=${{ github.repository }}
popd
env:
GH_TOKEN: ${{ github.token }}

- name: Checksums and sign
run: |
source icu4j/releases_tools/shared.sh
# Convert 76.1 to 76_1
underscore_version=$(echo $artifact_version | sed 's/\./_/g')
pushd ${RELEASE_FOLDER}
sha512sum -b icu4c* > SHASUM512.txt
md5sum -b *.jar > icu4j-${artifact_version}.md5
md5sum -b icu4c-*-data-bin-*.zip > icu4c-${underscore_version}-binary.md5
md5sum -b icu4c-*-src.* > icu4c-${underscore_version}-sources.md5
find . -type f -name 'icu4c*' -exec gpg --no-tty --batch --pinentry-mode loopback --passphrase=$MAVEN_GPG_PASSPHRASE -a --output {}.asc --detach-sig {} \;
gpg --no-tty --batch --pinentry-mode loopback --passphrase=$MAVEN_GPG_PASSPHRASE -a --output SHASUM512.txt.asc --detach-sig SHASUM512.txt
popd
env:
MAVEN_GPG_PASSPHRASE: ${{ secrets.MAVEN_GPG_PASSPHRASE }}

- name: Upload to release
run: |
gh release upload ${{ inputs.gitReleaseTag }} LICENSE --clobber --repo=${{ github.repository }}
gh release upload ${{ inputs.gitReleaseTag }} ${RELEASE_FOLDER}/*.md5 --clobber --repo=${{ github.repository }}
gh release upload ${{ inputs.gitReleaseTag }} ${RELEASE_FOLDER}/*.asc --clobber --repo=${{ github.repository }}
gh release upload ${{ inputs.gitReleaseTag }} ${RELEASE_FOLDER}/SHASUM512.txt --clobber --repo=${{ github.repository }}
env:
GH_TOKEN: ${{ github.token }}
89 changes: 89 additions & 0 deletions .github/workflows/release-icu4c-fedora.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
name: Release - ICU4C artifacts on Fedora

on:
workflow_dispatch:
inputs:
runTests:
description: 'Run the tests.'
type: boolean
default: true
gitReleaseTag:
description: 'Release tag to upload to. Must start with "release-"'
type: string

env:
RELEASE_FOLDER: '${{ github.workspace }}/releaseDist'

jobs:
build:
runs-on: ubuntu-latest
environment: release-env

container:
image: ghcr.io/${{ github.repository_owner }}/fedora-docker-gcr:latest
credentials:
username: ${{ secrets.DOCKER_CONTAINER_USER_NAME }}
password: ${{ secrets.DOCKER_CONTAINER_REGISTRY_TOKEN }}

permissions:
contents: write # So that we can upload to release

steps:

- name: Install gh (GitHub CLI)
run: |
# Don't install it in the docker image, get the latest (pros and cons)
dnf install -y gh

- name: Checkout and setup
uses: actions/checkout@v4
with:
lfs: true

- name: Config and build ICU4C proper
run: |
pushd icu4c/source
./runConfigureICU Linux/gcc
make -j8
popd

- name: Run tests
if: ${{ inputs.runTests }}
run: |
pushd icu4c/source
make check
popd

- name: Build release ICU4C
run: |
pushd icu4c/source
make DESTDIR=${RELEASE_FOLDER}/icu releaseDist
popd

- name: Collect artifacts in one folder
run: |
# Get the OS version in VERSION_ID
source /etc/os-release
# Get the ICU version in artifact_version
source icu4j/releases_tools/shared.sh
# Convert 76.1 to 76_1
underscore_version=$(echo $artifact_version | sed 's/\./_/g')
pushd ${RELEASE_FOLDER}
tar -czf icu4c-${underscore_version}-Fedora_Linux${VERSION_ID}-x64.tgz icu
rm -fr icu
popd

- name: Upload build results
uses: actions/[email protected]
with:
name: icu4c-fedora-binaries
path: ${{ env.RELEASE_FOLDER }}
retention-days: 3 # TBD if we want to keep them longer
overwrite: true

- name: Upload to release
if: ${{ inputs.gitReleaseTag && startsWith(inputs.gitReleaseTag, 'release-') }}
run: |
gh release upload ${{ inputs.gitReleaseTag }} ${RELEASE_FOLDER}/* --clobber --repo=${{ github.repository }}
env:
GH_TOKEN: ${{ github.token }}
Loading