[charts/gateway] otel update, system properties expansion, ocp routes

charts/gateway/Chart.yaml

@@ -2,7 +2,7 @@ apiVersion: v2
 appVersion: "11.1.1"
 description: This Helm Chart deploys the Layer7 Gateway in Kubernetes.
 name: gateway
-version: 3.0.30
+version: 3.0.31
 type: application
 home: https://github.com/CAAPIM/apim-charts
 maintainers:

charts/gateway/README.md

@@ -5,7 +5,7 @@ This Chart deploys the API Gateway v10.x onward with the following `optional` su
 The included MySQL subChart is enabled by default to make trying this chart out easier. ***It is not supported or recommended for production.*** Layer7 assumes that you are deploying a Gateway solution to a Kubernetes environment with an external MySQL database.
 
 ## Release notes
-- Current Chart Version 3.0.30
+- Current Chart Version 3.0.31
   - Please review release notes [here](./release-notes.md)
 
 ## Prerequisites
@@ -199,13 +199,16 @@ The following table lists the configurable parameters of the Gateway chart and t
 | `service.annotations`    | Additional annotations to add to the service               | {} |
 | `service.internalTrafficPolicy`    | [Internal Traffic Policy](https://kubernetes.io/docs/concepts/services-networking/service-traffic-policy/#using-service-internal-traffic-policy)               | `Cluster` |
 | `service.externalTrafficPolicy`    | [External Traffic Policy](https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/#preserving-the-client-source-ip)               | `Cluster` |
+
 | `ingress.enabled`    | Enable/Disable an ingress record being created               | `false` |
-| `ingress.annotations`    | Additional ingress annotations               | `{}` |
-| `ingress.hostname`    | Sets Ingress Hostname  | `nil` |
-| `ingress.port`    | The Gateway Port number/name to route to  | `8443` |
-| `ingress.tlsHostnames`    | Register additional Hostnames for the TLS Certificate  | `see values.yaml` |
-| `ingress.secretName`    | The name of an existing Cert secret, setting this does not auto-create the secret               | `tls-secret` |
-| `ingress.additionalHostnamesAndPorts`    | key/value pairs of hostname:port that will be added to the ingress object  | `see values.yaml` |
+| `ingress.openshift.route.enabled`    | Create an Openshift Route (Requires Openshift)               | `false` |
+| `ingress.openshift.route.wildcardPolicy`    | Openshift Route Wildcard Policy               | `None` |
+| `ingress.openshift.route.weight`    | Openshift Route Weight (0-255)               | `commented` |
+| `ingress.annotations`    | ingress annotations               | `{}` |
+| `ingress.labels`    | additional ingress labels               | `{}` |
+| `ingress.ingressClassName`    | Ingress Class Name               | `nginx` |
+| `ingress.tls`    | Ingress TLS Configuration               | `see values.yaml` |
+| `ingress.rules`    | Ingress Rules Configuration              | `see values.yaml` |
 | `startupProbe.enabled`    | Enable/Disable               | `false` |
 | `startupProbe.initialDelaySeconds`    | Initial delay               | `60` |
 | `startupProbe.timeoutSeconds`    | Timeout               | `1` |
@@ -508,25 +511,71 @@ config:
 ### Ingress Configuration
 The Gateway Helm Chart allows you to configure an Ingress Resource that your central Ingress Controller can manage. You can find more information on [Ingress Controllers](https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/) here.
 
-This represents the ingress configuration for Gateway Chart < 3.0.0 you need to configure an Ingress Resource for the API Gateway
+If your ingress controller is private and you would like to create an ingress record/route for the management service you can use the following configuration
+```
+ ...
+  rules:
+  - host: dev.ca.com <<== standard traffic
+    path: "/"
+    service:
+      port:
+        name: https
+  - host: dev-pm.ca.com <<== management traffic
+    path: "/"
+    backend: management <<== will target the management service
+    service:
+      port:
+        name: management
+```
 
+New Ingress Configuration Gateway Chart >= 3.0.31 (openshift route support)
 ```
 ingress:
-  enabled: true
-  annotations:
-  # Ingress class
-    kubernetes.io/ingress.class: nginx
-    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
+  # Set to true to create ingress object
+  enabled: false
+  # Set openshift.route.enabled to true if you are using Openshift and would like to use routes
+  openshift:
+    route:
+      enabled: false
+      wildcardPolicy: None
+    # weight: 100
+
+  # Ingress Class Name
+  ingressClassName: nginx
+  # Ingress labels (also apply to routes)
+  labels: {}
+  # Ingress annotations (also apply to routes)
+  annotations: {}
+  # nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
   # nginx.ingress.kubernetes.io/ssl-passthrough: "true"
-  secretName: tls-secret
-  hostname: dev.ca.com
-  tlsHostnames: []
-  # - dev.ca.com
-  # - dev1.ca.com
-  ## The port that you want to route to via ingress. This needs to be available via service.ports.
-  port: 8443
-  ## Define additional hostnames and ports as key-value pairs.
-  additionalHostnamesAndPorts: {}
+  # When the ingress is enabled, a host pointing to this will be created
+  tls:
+  - hosts:
+    - dev.ca.com
+    secretName: default
+  # - hosts:
+  #   - dev1.ca.com
+  #   secretName: default
+  rules:
+  - host: dev.ca.com
+    path: "/"
+    service:
+      port:
+        name: https
+      # number:
+  # - host: dev1.ca.com
+  #   path: "/"
+  #   service:
+  #     port:
+  #       name: https
+      # number:
+  # - host: dev-pm.ca.com
+  #   path: "/"
+  #   backend: management
+  #   service:
+  #     port:
+  #       name: management
+        # number:
 ```
 
 New Ingress Configuration Gateway Chart >= 3.0.0
@@ -567,6 +616,26 @@ ingress:
 #        #number:
 ```
 
+This represents the ingress configuration for Gateway Chart < 3.0.0 you need to configure an Ingress Resource for the API Gateway
+```
+ingress:
+  enabled: true
+  annotations:
+  # Ingress class
+    kubernetes.io/ingress.class: nginx
+    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
+  # nginx.ingress.kubernetes.io/ssl-passthrough: "true"
+  secretName: tls-secret
+  hostname: dev.ca.com
+  tlsHostnames: []
+  # - dev.ca.com
+  # - dev1.ca.com
+  ## The port that you want to route to via ingress. This needs to be available via service.ports.
+  port: 8443
+  ## Define additional hostnames and ports as key-value pairs.
+  additionalHostnamesAndPorts: {}
+```
+
 [Back to Additional Guides](#additional-guides)
 
 ### PM Tagger Configuration
@@ -610,11 +679,27 @@ OpenTelemetry is configured on the Gateway in two places, system properties and
 
 These can be configured in values.yaml. See the section below to view examples of how and where to configure this.
 
+- config.otel
+```
+config:
+  ...
+  otel:
+    # If sdkOnly is enabled we will inject the above environment variables
+    # Note that this is container level configuration only. You will still need to set the relevant cluster-wide and system properties below
+    sdkOnly:
+      enabled: true
+    # Used to inject additional resource attributes for tracking with the sdkOnly approach
+    # these can then be used as an additional filter in your observability backend
+    additionalResourceAttributes:
+    - test=someEnvValue
+   # - test1=someEnvValue1
+```
+
+
 - system.properties
 ```
 otel.sdk.disabled=false
 otel.java.global-autoconfigure.enabled=true
-otel.service.name=ssg-gateway
 otel.exporter.otlp.endpoint=http://localhost:4318/
 otel.exporter.otlp.protocol=http/protobuf
 otel.traces.exporter=otlp

charts/gateway/production-values.yaml

@@ -186,6 +186,24 @@ config:
   # If you are using an earlier version of the Gateway, these will be ignored.
   # minHeapSize: "1g"
   # maxHeapSize: "3g"
+  # The OTel SDK uses the following environment variables to gather information about the container
+  # NODE_NAME - Kubernetes Node
+  # POD_NAME - Podname, also hostname
+  # NAMESPACE
+  # CONTAINER_NAME - this is always gateway
+  # OTEL_SERVICE_NAME - <release-name>-<chart-name>
+  # OTEL_RESOURCE_ATTRIBUTES
+  # When using auto-instrumentation (injecting the OTel Java Agent via the OpenTelemetryOperator) these values are automatically set
+  # When using the sdk only approach (no OTel Java Agent) we set these using built-in metadata fields
+  otel:
+    # If sdkOnly is enabled we will inject the above environment variables
+    # Note that this is container level configuration only. You will still need to set the relevant cluster-wide and system properties below
+    sdkOnly:
+      enabled: false
+    # Used to inject additional resource attributes for tracking with the sdkOnly approach
+    additionalResourceAttributes: []
+    # - test=someEnvValue
+    # - test1=someEnvValue1
   javaArgs:
     - -Dcom.l7tech.bootstrap.autoTrustSslKey=trustAnchor,TrustedFor.SSL,TrustedFor.SAML_ISSUER
     - -Dcom.l7tech.server.audit.message.saveToInternal=false
@@ -268,13 +286,17 @@ config:
     # If you would like to use the built in OpenTelemetry SDK uncomment and set the following configuration
     # otel.sdk.disabled=false
     # otel.java.global-autoconfigure.enabled=true
-    # otel.service.name=ssg-gateway
     # otel.exporter.otlp.endpoint=http://localhost:4318/
     # otel.exporter.otlp.protocol=http/protobuf
     # otel.traces.exporter=otlp
     # otel.metrics.exporter=otlp
     # otel.logs.exporter=none
     # Additional properties go here
+  # Additional System properties are appended at the end of system.properties
+  # Defined as key/value pairs
+  additionalSystemProperties: []
+  # - name: test
+  #   value: test123
 
  # If enabled this will override the default listen ports and their configuration in the API Gateway
   listenPorts:
@@ -885,39 +907,46 @@ otk:
 ingress:
   # Set to true to create ingress object
   enabled: true
+  # Set openshift.route.enabled to true if you are using Openshift and would like to use routes
+  openshift:
+    route:
+      enabled: false
+      wildcardPolicy: None
+    # weight: 100
   # Ingress Class Name
   ingressClassName: nginx
   # Ingress annotations
   annotations:
-  # Ingress class
-   # kubernetes.io/ingress.class: nginx
     nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
   # nginx.ingress.kubernetes.io/ssl-passthrough: "true"
   # When the ingress is enabled, a host pointing to this will be created
-  # By default clusterHostname is used, only set this if you want to use a different host
-   ## Enable TLS configuration for the hostname defined at ingress.hostname/clusterHostname parameter
   tls:
   - hosts:
     - dev.ca.com
     secretName: default
   # - hosts:
   #   - dev1.ca.com
   #   secretName: default
-
   rules:
   - host: dev.ca.com
     path: "/"
     service:
       port:
         name: https
       # number:
-  #  - host: dev1.ca.com
-  #    path: "/"
-  #    backend: management
-  #    service:
-  #      port:
-  #        name: management
-  #        #number:
+  # - host: dev1.ca.com
+  #   path: "/"
+  #   service:
+  #     port:
+  #       name: https
+      # number:
+  # - host: dev-pm.ca.com
+  #   path: "/"
+  #   backend: management
+  #   service:
+  #     port:
+  #       name: management
+        # number:
 
 # Additional Environment variables to be added to the Gateway Configmap
 additionalEnv: {}

charts/gateway/release-notes.md

@@ -7,8 +7,84 @@ The Layer7 API Gateway is now running with Java 17 with the release of v11.1.00.
 
 If you use Policy Manager, you will need to update to v11.1.00.
 
+## 3.0.31 General Updates
+- Support for Openshift Routes (disabled by default)
+  - Uses passthrough termination (tls only)
+    - path is ignored in this mode
+  - Converts existing ingress
+    - will create a route for each ingress rule
+  - Management service can be routed
+
+To enable - see [ingress configuration](./README.md#ingress-configuration) for more details
+```
+ingress:
+  enabled: true
+  openshift:
+    route:
+      enabled: true
+      wildcardPolicy: None
+    # weight: 100
+  ...
+  rules:
+  - host: dev.ca.com
+    path: "/"
+    service:
+      port:
+        name: https
+  - host: dev1.ca.com
+    path: "/"
+    service:
+      port:
+        name: https
+  - host: dev-pm.ca.com
+    path: "/"
+    backend: management
+    service:
+      port:
+        name: management
+```
+
+- New way to add system properties
+  - You can now use key/value pairs to extend [system properties](./README.md#system-properties)
+  - No impact to existing configuration
+```
+config:
+  ...
+  additionalSystemProperties:
+  - name: test
+    value: test123
+```
+- New Deployment Configuration Options for the OTel SDK Only approach (Disabled by default)
+  - Does ***NOT*** configure system or cluster-wide properties, this step is still required
+  - Requires a Gateway restart when enabled
+  - injects the following environment variables which are then used to set OTEL_RESOURCE_ATTRIBUTES
+    - NODE_NAME      ==> spec.nodeName
+    - POD_NAME       ==> metadata.name
+    - NAMESPACE      ==> metadata.namespace 
+    - CONTAINER_NAME ==> gateway
+    - OTEL_SERVICE_NAME - `<release-name>-<chart-name>`
+    - OTEL_RESOURCE_ATTRIBUTES ==> custom values can be set with config.otel.additionalResourceAttributes
+      - defaults (if config.otel.sdkOnly.enabled is true)
+        - service.name         ==> OTEL_SERVICE_NAME
+        - service.version      ==> .Values.image.tag
+        - k8s.container.name   ==> gateway
+        - k8s.deployment.name  ==> OTEL_SERVICE_NAME
+        - k8s.namespace.name   ==> NAMESPACE
+        - k8s.node.name        ==> NODE_NAME
+        - k8s.pod.name         ==> POD_NAME
+```
+config:
+...
+  otel:
+    sdkOnly:
+      enabled: true
+    # Used to inject additional resource attributes for tracking with the sdkOnly approach
+    additionalResourceAttributes:
+    - test=someEnvValue
+    - test1=someEnvValue1
+```
+
 ## 3.0.30 General Updates
-Release notes will also be moved to a new file before merge...
 **Note** Gateway restart required if using preview Redis features.
 - Support added for running the Gateway without [Diskless Config](./README.md#diskless-configuration)
   - Uses node.properties which can be mounted via [Secret or Secret Store CSI Driver](https://secrets-store-csi-driver.sigs.k8s.io/)

charts/gateway/templates/_helpers.tpl

@@ -203,6 +203,21 @@ Define OTK Image Pull Secret Name
 {{- end -}}
 {{- end -}}
 
+
+{{/*
+ Define OTEL_RESOURCE_ATTRIBUTES Environment variable
+ */}}
+{{- define "gateway.otel.resource.attributes" -}}
+{{ $resourceAttributes := printf "%s,service.version=%s" "k8s.container.name=$(CONTAINER_NAME),k8s.deployment.name=$(OTEL_SERVICE_NAME),service.name=$(OTEL_SERVICE_NAME),k8s.namespace.name=$(NAMESPACE),k8s.node.name=$(NODE_NAME),k8s.pod.name=$(POD_NAME)" .Values.image.tag }}
+{{- if and (.Values.config.otel.sdkOnly.enabled) (.Values.config.otel.additionalResourceAttributes) -}}
+ {{- $additionalResourceAttributes := join "," .Values.config.otel.additionalResourceAttributes }}
+ {{- printf "%s,%s" $resourceAttributes $additionalResourceAttributes -}}
+{{- else -}}
+    {{- printf "%s" $resourceAttributes -}}
+{{- end -}}
+{{- end -}}
+
+
 {{/*
  Validate OTK installation type (SINGLE, INTERNAL, DMZ)
 */}}