Multistage docker building system

[germa89]

Writing this here so it can be open to everybody.

Context

While thinking on how to improve the current build system for the ubuntu docker image, I came up with the idea of using multistage builds. Our current image build system depends too much on github runners (it is not practical to build it locally); using this, it can be more easy to build containers locally.

Why improve the building process?

Several reasons:

• It is not replicable locally. Right now we depend too much on Github for that. We need to be able to generate a docker image locally. Having a multistage docker file allow us put in there all the logic regarding installing and manipulating MAPDL. Additionally, it provides layers/images caching so the building could be faster when iterating.
• We should avoid fragmentation between the docker building documentation (in this repo) and the actual docker files (in another internal repo). We should consider the option of either...
  • Pull the docker image from pymapdl repo.
  • Open the other repo so we leave the real docker file available to everybody.
• Better parametrization. We need docker images with different MPI implementations, tools (python, git, etc), ... Multistage building allow us to generate a default stage... and then remove (MAPDL files)/add (tools) stuff into it to generate other docker images.
• We need to sort out the tagging/releasing of docker images. We have right now a parallel process, which is fine. But we are forced to handle several docker files. And we are not correctly having releases which can clarify which are the changes in each docker image.

Todo

I need to document this and implement it internally in our internal repo.

Pinging @dts12263 @koubaa for visibility.

Logic

Following the current implementation where we install MAPDL first, and then copy the files to the building context, we can do similar.
We will have two stages:

1. builder stage where we copy the installation files and install MAPDL.
2. container stage where we copy the installed files to.

Process

Download the ansys installation files from customer portal, and locate them in the working directory:

.dockerignore
Dockerfile.multistage
STRUCTURES_2024R2_LINX64
    242-1.dvd
    INSTALL
    LICENSE.TXT
    aas
    acp
    ...

And having the following docker file Dockerfile.multistage:

# We can add ARG here

# Building builder stage, where MAPDL is installed
FROM ubuntu:22.04 AS builder

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get -y update && apt-get install -y curl \
    libgl1 \
    libglu1 \
    libxcb-icccm4 \
    libxcb-image0 \
    libxcb-keysyms1 \
    libxcb-randr0 \
    libxcb-render-util0 \
    libxcb-render0 \
    libxcb-shape0 \
    libxcb-shm0 \
    libxcb-sync1 \
    libxcb-util1 \
    libxcb-xfixes0 \
    libxcb-xinerama0 \
    libxcb-xkb1 \
    libxi6 \
    libxkbcommon-x11-0 \
    libxkbcommon0 \
    libxm4 \
    openssh-client \
    libglib2.0-0


COPY ./STRUCTURES_2024R2_LINX64 /MAPDL_INSTALLATION
RUN chmod +x /MAPDL_INSTALLATION/INSTALL && \
    mkdir /ansys_inc/ && \
    sh /MAPDL_INSTALLATION/INSTALL \
      -install_dir /ansys_inc \
      -nochecks -mechapdl -ansyscust -silent

# Final container
FROM ubuntu:22.04 AS container

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update && \
    apt-get install -y \
      sudo \
      git \
      python3.10-venv \
      python3-pip \
      openssh-client \
      libgomp1 \
      libgl1 \
      libglu1 \
      libxm4 \
      libxi6 && \
    apt install -y software-properties-common \
    && add-apt-repository -y ppa:zeehio/libxp \
    && apt-get update \
    && apt-get install -y libxp6 && \
    mkdir -p /ansys_inc

# Copy files from previous stage
COPY --from=builder /ansys_inc /ansys_inc

You can build a working image `` using:

docker build -t test_multi -f Dockerfile.multistage  .

If you are building on a different architecture (for instance you are using a MacOS MX to build a container Intel/AMD arch based), you need to add --platform="linux/amd64":

docker build -t test_multi -f Dockerfile.multistage --platform="linux/amd64" .

You can test it with docker run -it test_multi /bin/bash.

Notes/Caveats

• The .dockerignore file will have to be applieds as a shell script:

  xargs -a ./.dockerignore rm -rf
  

• I haven't tested if the created image works on AMD arch. It should though.

[germa89]

We should implement a check in our CICD that makes it fail if the size of the layer is much.

#!/bin/bash

# The name or ID of the Docker image
IMAGE_NAME="$1"

# Size limit in bytes (5 GB = 5 * 1024^3 bytes)
SIZE_LIMIT=$((5 * 1024**3))

# Get the size of the Docker image in bytes
IMAGE_SIZE=$(docker image inspect "$IMAGE_NAME" --format='{{.Size}}')

# Convert bytes to GB for display
IMAGE_SIZE_GB=$(echo "scale=2; $IMAGE_SIZE / (1024^3)" | bc)

# Check if the image size exceeds the limit
if [ "$IMAGE_SIZE" -gt "$SIZE_LIMIT" ]; then
    echo "The Docker image '$IMAGE_NAME' exceeds 5 GB (size: ${IMAGE_SIZE_GB} GB)."
    exit 1
else
    echo "The Docker image '$IMAGE_NAME' is within the size limit (size: ${IMAGE_SIZE_GB} GB)."
    exit 0
fi

Useful command to get the size per layer...

docker history "$IMAGE_NAME"

Reference: https://gist.github.com/MichaelSimons/fb588539dcefd9b5fdf45ba04c302db6

Gist

Retrieving Docker Image Sizes

Retrieving Docker Image Sizes. GitHub Gist: instantly share code, notes, and snippets.