Merge pull request #338 from CAAPIM/portal_de620051

[charts/portal] Race condition - DE620051
CAAPIM · Nov 21, 2024 · 2e5206e · 2e5206e
2 parents 19ac936 + cffd884
commit 2e5206e
Show file tree

Hide file tree

Showing 6 changed files with 34 additions and 4 deletions.
diff --git a/charts/portal/Chart.yaml b/charts/portal/Chart.yaml
@@ -2,14 +2,13 @@ apiVersion: v2
 appVersion: "5.3.1"
 description: CA API Developer Portal
 name: portal
-version: 2.3.10
+version: 2.3.11
 type: application
 home: https://github.com/CAAPIM/apim-charts
 maintainers:
   - name: Gazza7205
 sources:
   - https://github.com/CAAPIM/apim-charts
-engine: gotpl
 dependencies:
 - name: druid
   version: ^1.0.0

diff --git a/charts/portal/README.md b/charts/portal/README.md
@@ -3,6 +3,25 @@ The Layer7 API Developer Portal (API Portal) is part of the Layer7 API Managemen
 
 ## Introduction
 This Chart deploys the Layer7 API Developer Portal on a Kubernetes Cluster using the Helm Package Manager.
+
+## Release Notes
+
+## 2.3.11 General Updates
+- Added a preflight check (initContainer) for the core apim/ingress deployment
+  - This resolves a race condition that occurs on slower hardware where apim/ingress starts before other dependent services are ready. 
+  - This is ***enabled by default***.
+    - This only gets added when you install the Chart.
+      - A Helm upgrade will ***NOT*** restart the apim deployment if you have already installed the Chart and are upgrading to this version.
+      - A Helm upgrade will restart the apim deployment if you installed from this version and upgrade the Chart.
+    - If you wish to disable this, set apim.preflightCheck.enabled to false
+    ```
+    apim:
+      ...
+      preflightCheck:
+        enabled: true
+      ...
+    ```
+
 ## 2.3.10 General Updates
 - This new version of the chart supports API Portal 5.3.1
 - Removed PSSG container
@@ -302,6 +321,7 @@ This section describes configurable parameters in **values.yaml**, there is also
 | `analytics.affinity`                 | Affinity for pod assignment                                  | `{} evaluated as a template`                                 |
 | `analytics.additionalLabels`         | A list of custom key: value labels                           | `not set`                                                    |
 | `apim.forceRedeploy`                 | Force redeployment during helm upgrade whether there is a change or not | `false`                                                      |
+| `apim.preflightCheck.enabled`        | Resolves a race condition that occurs on slower hardware where apim/ingress starts before other dependent services are ready |`true`                                                      |
 | `apim.replicaCount`                  | Number of APIM nodes                                         | `1`                                                          |
 | `apim.image.pullPolicy`              | APIM image pull policy                                       | `IfNotPresent`                                               |
 | `apim.otkDb.name`                    | APIM OTK Database name                                       | `otk_db`                                                     |
@@ -979,7 +999,6 @@ If the RabbitMQ cluster is stopped or removed out of order, there is a chance th
 $ helm upgrade <release-name> --set-file <values-from-install> --set <values-from-install> -f <my-values.yaml> layer7/portal
 ```
 
-
 ### Helm UPGRADE FAILED: cannot patch "db-upgrade" and "rbac-upgrade"
 If helm upgrade of the portal fails with error "Error: UPGRADE FAILED: cannot patch 'db-upgrade'", it is becasue of the limitation in kubernetes where a job can not be updated.
 
@@ -994,7 +1013,6 @@ kubectl delete job db-upgrade -n <nameSpace>
 kubectl delete job rbac-upgrade -n <nameSpace>
 ```
 
-
 ### MySQL container in unhealthy state
 
 #### The chart was uninstalled and re-installed

diff --git a/charts/portal/charts/druid-1.0.14.tgz b/charts/portal/charts/druid-1.0.14.tgz
diff --git a/charts/portal/templates/apim/apim-deployment.yaml b/charts/portal/templates/apim/apim-deployment.yaml
@@ -55,6 +55,15 @@ spec:
         - name: liquidbase-upgrade-rbac
           image: "{{ .Values.global.portalRepository }}{{ .Values.image.upgradeVerify }}"
           args: ["job", 'rbac-upgrade']
+        {{- if and (.Values.apim.preflightCheck.enabled) (.Release.IsInstall) }}
+        - name: apim-core-preflight-check
+          image: "{{ .Values.global.portalRepository }}{{ .Values.image.upgradeVerify }}"
+          command: ["/bin/sh","-c"]
+          args:
+          - sleep 120;
+            if [[ "$(curl -w '%{http_code}' -s -o /dev/null rabbitmq:15672)" != "200" ]]; then echo "broker not ready" && exit 1;fi;
+            if [[ "$(curl -w '%{http_code}' -s -o /dev/null http://authenticator:9081/v1/echo;)" != "200" ]]; then echo "authenticator not ready" && exit 1;fi;
+        {{- end }}
       containers:
         - name: apim
           image: "{{ .Values.global.portalRepository }}{{ .Values.image.apim }}"

diff --git a/charts/portal/values-production.yaml b/charts/portal/values-production.yaml
@@ -217,6 +217,8 @@ analytics:
 
 apim:
   forceRedeploy: false
+  preflightCheck:
+    enabled: true
   replicaCount: 2
   image:
     pullPolicy: IfNotPresent

diff --git a/charts/portal/values.yaml b/charts/portal/values.yaml
@@ -210,6 +210,8 @@ analytics:
 
 apim:
   forceRedeploy: false
+  preflightCheck:
+    enabled: true
   replicaCount: 1
   image:
     pullPolicy: IfNotPresent