Improved DevOps: new GitHub Actions CI/CD system and refined Docker a…

…nd server-frontend integration

Numerous modifications have been made to improve the deployment of VSCode4Teaching and adapt it to the presence of the new web application:

- Changed system for continuous integration, deployment and delivery (CI/CD), moving away from using Travis CI to start using GitHub Actions. For this, two workflows have been defined: one for the extension and one for the server & Angular web application. Specific documentation about the new system is introduced in the .github/workflows folder.
- The Dockerfile, docker-compose.yml and .env files have been modified and moved to the root of the project, since they affect two of the three components. In addition, the versions of the images used for the compilation in the Dockerfile have been updated and the deployment using Compose has been updated for v2.
- The .dockerignore files for each component have been removed to introduce a single description in the project root.
- The server has been modified to introduce a new request interceptor that, before resolving the routing in the framework, decides if the received path should be handled by the web application (redirecting the request to índex.html) or if it should be resolved in the framework. This allows a better integration of the compiled Angular framework as a static resource in the application and, thus, the presence of the /app path to access all Angular resources can be eliminated.

.dockerignore

@@ -1 +1,43 @@
-.git
+# Ignore VSCode4Teaching unnecesary directories
+**/v4t-course/
+
+# Ignore Maven build directories
+**/target/
+**/logs/
+**/tmp/
+
+# Ignore Node/Angular build directories and dependencies
+**/node_modules/
+**/dist/
+**/.angular/
+**/.cache/
+
+# Ignore IDE specific directories and files
+**/.vscode/
+**/.idea/
+**/.project
+**/.classpath
+
+# Ignore documentation files
+**/*.md
+
+# Ignore MacOS specific files
+**/.DS_Store
+**/.AppleDouble
+**/.LSOverride
+
+# Ignore other unnecesary system files
+**/Thumbs.db
+**/ehthumbs.db
+**/Desktop.ini
+**/$RECYCLE.BIN/
+
+# Ignore git files
+**/.git/
+**/.gitignore
+
+# Ignore Dockerfile and Dockerignore themselves
+**/docker-compose.yml
+**/compose.yml
+**/Dockerfile
+**/.dockerignore

vscode4teaching-server/docker/.env → .env

@@ -1,16 +1,22 @@
 SERVER_PORT=8080
+
 MYSQL_ROOT_PASSWORD=T4cwGK3q5NdR3vMz
 MYSQL_DATABASE=vscode4teach
+
 SPRING_DATASOURCE_USERNAME=vscode4teach
 SPRING_DATASOURCE_PASSWORD=AvScnGQp5e4GHnd
 SPRING_DATASOURCE_URL=jdbc:mysql://db/vscode4teach?useUnicode=true&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=UTC
+
 JWT_SECRET=vscode4teaching
+
 V4T_FILEDIRECTORY=/app/v4t_courses
+
 FILE_INITIALIZATION=false
 DATA_INITIALIZATION=false
 SPRING_JPA_HIBERNATE_DDL_AUTO=create-drop
+
 SUPERUSER_USERNAME=admin
-SUPERUSER_PASSWORD=admin
+SUPERUSER_PASSWORD=adminpassword
 SUPERUSER_EMAIL=[email protected]
 SUPERUSER_NAME=Admin
 SUPERUSER_LASTNAME=Admin

.github/workflows/README.md

@@ -0,0 +1,26 @@
+# VSCode4Teaching
+**Developer documentation on the Continuous Integration, Delivery and Deployment (*CI/CD*) system of this project**
+
+This project is configured with a continuous integration, deployment and delivery system through [GitHub Actions](https://github.com/features/actions). This system comprises two *workflows*: one for the extension (file [`extension.ci.yml`](extension.ci.yml)) and one for the server and web application (file [`server.ci.yml`](server.ci.yml)).
+
+### Detailed description of declared jobs
+The aforementioned files define the jobs that will be executed sequentially based on the type of changes registered in the repository. These jobs are:
+- `test`. This job runs when changes are pushed to the `master` (or `main`) and `develop` branches or when a *pull request* is opened targeting any of these branches. After setting up the execution environment on the pipeline runner, it executes the complete test suite implemented in the component on which the pipeline is launched, returning success if the tests pass.
+- `publish`. This job runs when a change is made to the `master` branch, meaning a new version of VSCode4Teaching is being released. This step requires the deployed version to be correctly configured in each component's manifest, which are POM (`pom.xml`) for the server, and `package.json` for both the extension and web application.
+    - For the extension, this job will compile and package the extension source code into VSIX format and deploy it to the [Visual Studio Code Marketplace](https://marketplace.visualstudio.com/VSCode) in its new version. This job will fail if a version with the same identifier has already been deployed previously or if the authentication token has expired or is invalid for publishing.
+    - For the server, it will execute a Docker image build process defined through the `Dockerfile` file, which is *multistage*: it will build the web application in a first Node container to produce the static files (HTML, CSS, JS) needed to run the application without Node, compile the server source code along with the frontend static resources in a second Maven container with JDK 11, and use a third container to define the final image on JDK 11, setting the *entrypoint* command to run the JAR obtained in the previous stage. The resulting image is tagged with the version declared in the POM and also as `latest`. Both images are pushed to the specified [Docker Hub](https://hub.docker.com) repository.
+- `deploy`. Executed only under the same conditions as `publish`, and declared just in the server workflow, this job allows deploying the built Docker image to the defined production server. It connects via SSH to the remote computer, copies the `docker-compose.yml` file, pulls the new image generated in the previous job and restarts the Docker Compose instance running the configured production server.
+
+### Parameterized definition of variables and secrets
+To run the previously explained jobs, certain variables and secrets must be properly defined in the repository:
+- Publishing the server image to Docker Hub. These variables are used during the server (embedding the built web application) image publishing to Docker Hub and subsequent production deployment.
+    - `DOCKER_HUB_USERNAME` (variable). The username used to create the Docker image for the VSCode4Teaching server (and web application as static frontend resource). It will be the first part of the image name (preceding the `/`) and must match the name specified in the `docker-compose.yml` file to ensure successful production deployment.
+    - `DOCKER_HUB_IMAGE_NAME` (variable). Properly, the name of the image (following the `/` that separates it from the username). It must match the name used in the Docker Compose definition to complete the deployment job correctly.
+    - `DOCKER_HUB_PAT` (secret). The personal access token required to publish the built image to Docker Hub. If it is incorrect or invalid, the pipeline execution will fail.
+- Deploying on the production server. These variables are used to access the remote computer that hosts the production server.
+    - `EDUKAFORA_HOST` (variable). The address to which the pipeline executor will connect via SSH during deployment through the default port (22).
+    - `EDUKAFORA_USER` (variable). The user hosting the VSCode4Teaching project on the production machine.
+    - `EDUKAFORA_SSH_KEY` (secret). The private key used to authenticate the SSH connection.
+    - `EDUKAFORA_PATH` (variable). The path, preferably absolute, on the production environment where the existing deployment is located and will be replaced.
+- Publishing the extension to the Visual Studio Code Marketplace. This variable is used in the extension workflow for publishing new versions.
+    - `VSCODE_MARKETPLACE_PAT` (secret). The personal access token required to authenticate the publication to the Visual Studio Code Marketplace. It has a scheduled expiration, so it may need to be replaced or renewed if an error occurs during the pipeline execution. The definition of the extension name, publisher, and other deployment and publication details are contained in its `package.json`, and the configured token must be able to authenticate and published according to the parameters set in the manifest file.

.github/workflows/extension.ci.yml

@@ -0,0 +1,61 @@
+name: "Extension pipeline"
+
+on:
+  push:
+    branches:
+      - master
+      - main
+      - develop
+  pull_request:
+    branches:
+      - master
+      - main
+      - develop
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: ./vscode4teaching-extension
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.ref }}
+      - name: Set up Node version
+        uses: actions/setup-node@v4
+        with:
+          node-version: 18
+      - name: Install dependencies
+        run: npm ci
+      - name: Build
+        run: npm run build
+      - name: Test
+        run: npm run test
+  publish:
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: ./vscode4teaching-extension
+    if: ${{ github.event_name == 'push' && (github.ref == 'refs/heads/master' || github.ref == 'refs/heads/main') }}
+    needs: test
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.ref }}
+      - name: Set up Node version
+        uses: actions/setup-node@v4
+        with:
+          node-version: 18
+      - name: Install dependencies and VSCode extension manager (vsce)
+        run: npm ci && npm install -g @vscode/vsce
+      - name: Check VSCode Marketplace token validity
+        run: npx vsce verify-pat
+        env:
+          VSCE_PAT: ${{ secrets.VSCODE_MARKETPLACE_PAT }}
+      - name: Publish to VSCode Marketplace
+        run: npx vsce publish --allow-star-activation
+        env:
+          VSCE_PAT: ${{ secrets.VSCODE_MARKETPLACE_PAT }}

.github/workflows/server.ci.yml

@@ -0,0 +1,108 @@
+name: "Server pipeline"
+
+on:
+  push:
+    branches:
+      - master
+      - main
+      - develop
+  pull_request:
+    branches:
+      - master
+      - main
+      - develop
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: ./vscode4teaching-server
+    steps:
+    - name: Checkout repository
+      uses: actions/checkout@v4
+    - name: Set up Java version
+      uses: actions/setup-java@v4
+      with:
+        java-version: 11
+        distribution: temurin
+    - name: Test
+      run: ./mvnw clean dependency:resolve test
+  publish:
+    runs-on: ubuntu-latest
+    if: ${{ github.event_name == 'push' && (github.ref == 'refs/heads/master' || github.ref == 'refs/heads/main') }}
+    needs: test
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.ref }}
+      - name: Get deployed version from POM
+        run: |
+          VERSION=$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout)
+          echo VERSION=$VERSION >> $GITHUB_ENV
+        working-directory: ./vscode4teaching-server
+      - name: Build Docker image
+        run: |
+          env | grep DOCKER
+          DOCKER_IMAGE="$DOCKER_HUB_USERNAME/$DOCKER_HUB_IMAGE_NAME"
+          echo DOCKER_IMAGE=$DOCKER_IMAGE >> $GITHUB_ENV
+          docker build -t $DOCKER_IMAGE:$VERSION -t $DOCKER_IMAGE:latest .
+        env:
+          DOCKER_HUB_USERNAME: ${{ vars.DOCKER_HUB_USERNAME }}
+          DOCKER_HUB_IMAGE_NAME: ${{ vars.DOCKER_HUB_IMAGE_NAME }}
+          VERSION: ${{ env.VERSION }}
+      - name: Push to Docker Hub
+        run: |
+          echo $DOCKER_HUB_PAT | docker login -u $DOCKER_HUB_USERNAME --password-stdin
+          docker push $DOCKER_IMAGE:$VERSION
+          docker push $DOCKER_IMAGE:latest
+        env:
+          DOCKER_HUB_USERNAME: ${{ vars.DOCKER_HUB_USERNAME }}
+          DOCKER_IMAGE: ${{ env.DOCKER_IMAGE }}
+          VERSION: ${{ env.VERSION }}
+          DOCKER_HUB_PAT: ${{ secrets.DOCKER_HUB_PAT }}
+
+  deploy:
+    runs-on: ubuntu-latest
+    if: ${{ github.event_name == 'push' && (github.ref == 'refs/heads/master' || github.ref == 'refs/heads/main') }}
+    needs: publish
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.ref }}
+      - name: Setup SSH agent to connect to deploy server
+        run: |
+          mkdir -p ~/.ssh
+          echo "${EDUKAFORA_SSH_KEY}" > ~/.ssh/id_ed25519
+          chmod 600 ~/.ssh/id_ed25519
+          ssh-keyscan -t ed25519 ${EDUKAFORA_HOST} >> ~/.ssh/known_hosts
+        env:
+          EDUKAFORA_SSH_KEY: ${{ secrets.EDUKAFORA_SSH_KEY }}
+          EDUKAFORA_HOST: ${{ vars.EDUKAFORA_HOST }}
+      - name: Prepare new VSCode4Teaching version
+        run: |
+          scp docker-compose.yml ${EDUKAFORA_USER}@${EDUKAFORA_HOST}:${EDUKAFORA_PATH}/compose-new.yml
+          ssh ${EDUKAFORA_USER}@${EDUKAFORA_HOST} "cd ${EDUKAFORA_PATH} && docker compose -f compose-new.yml pull --policy missing"
+        env:
+          EDUKAFORA_HOST: ${{ vars.EDUKAFORA_HOST }}
+          EDUKAFORA_USER: ${{ vars.EDUKAFORA_USER }}
+          EDUKAFORA_PATH: ${{ vars.EDUKAFORA_PATH }}
+      - name: Stop VSCode4Teaching and change by new version
+        run: |
+          ssh ${EDUKAFORA_USER}@${EDUKAFORA_HOST} "cd ${EDUKAFORA_PATH} && docker compose down"
+          ssh ${EDUKAFORA_USER}@${EDUKAFORA_HOST} "cd ${EDUKAFORA_PATH} && mv compose.yml compose-old.yml && mv compose-new.yml compose.yml"
+          ssh ${EDUKAFORA_USER}@${EDUKAFORA_HOST} "cd ${EDUKAFORA_PATH} && docker compose up -d"
+        env:
+          EDUKAFORA_USER: ${{ vars.EDUKAFORA_USER }}
+          EDUKAFORA_HOST: ${{ vars.EDUKAFORA_HOST }}
+          EDUKAFORA_PATH: ${{ vars.EDUKAFORA_PATH }}
+      - name: Clean up old version and SSH keys
+        run: |
+          ssh ${EDUKAFORA_USER}@${EDUKAFORA_HOST} "cd ${EDUKAFORA_PATH} && rm compose-old.yml"
+          rm -r ~/.ssh
+        env:
+          EDUKAFORA_USER: ${{ vars.EDUKAFORA_USER }}
+          EDUKAFORA_HOST: ${{ vars.EDUKAFORA_HOST }}
+          EDUKAFORA_PATH: ${{ vars.EDUKAFORA_PATH }}

.gitignore

@@ -0,0 +1,2 @@
+**/.DS_Store
+**/*.env*

Dockerfile

@@ -1,23 +1,20 @@
 # Step 1: Compilation of Angular frontend
 # It will be embedded as a static resource into Spring Boot backend
-FROM node:16.13.2 AS angular
+FROM node:18 AS angular
 COPY vscode4teaching-webapp /usr/src/app
 WORKDIR /usr/src/app
 RUN ["npm", "install"]
 RUN ["npm", "run", "build"]
 
 # Step 2: Compilation of Maven project (generation of JAR)
-FROM maven:3.8.4-jdk-11 AS builder
+FROM maven:3.9.7-eclipse-temurin-11 AS builder
 COPY vscode4teaching-server /data
-COPY --from=angular /usr/src/app/dist /data/src/main/resources/static
+COPY --from=angular /usr/src/app/dist/vscode4teaching /data/src/main/resources/static/
 WORKDIR /data
 RUN ["mvn", "clean", "package"]
 
 # Step 3: Generation of Docker image using the JAR previously built
-FROM adoptopenjdk/openjdk11:latest
-RUN apt-get update && apt-get install -y netcat && rm -rf /var/lib/apt/lists/*
-COPY --from=builder /data/target/vscode4teaching-server-*.jar ./app/vscode4teaching-server-*.jar
-COPY vscode4teaching-server/docker/waitDB.sh ./app/waitDB.sh
+FROM eclipse-temurin:11
+COPY --from=builder /data/target/vscode4teaching-server-*.jar ./app/vscode4teaching-server.jar
 EXPOSE 8080
-RUN ["chmod", "+x", "./app/waitDB.sh"]
-CMD ["./app/waitDB.sh"]
+ENTRYPOINT [ "java", "-jar", "./app/vscode4teaching-server.jar" ]