Skip to content

Commit

Permalink
Merge pull request #2813 from jdgeisler/add-stateful-metrics-generator
Browse files Browse the repository at this point in the history
[tempo-distributed] add optional statefulset and traces_storage config for metrics generator
  • Loading branch information
mapno authored Feb 14, 2024
2 parents 06500b7 + c7720dd commit e880312
Show file tree
Hide file tree
Showing 5 changed files with 187 additions and 4 deletions.
2 changes: 1 addition & 1 deletion charts/tempo-distributed/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ apiVersion: v2
name: tempo-distributed
description: Grafana Tempo in MicroService mode
type: application
version: 1.8.3
version: 1.8.4
appVersion: 2.3.1
engine: gotpl
home: https://grafana.com/docs/tempo/latest/
Expand Down
10 changes: 8 additions & 2 deletions charts/tempo-distributed/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# tempo-distributed

![Version: 1.8.3](https://img.shields.io/badge/Version-1.8.3-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 2.3.1](https://img.shields.io/badge/AppVersion-2.3.1-informational?style=flat-square)
![Version: 1.8.4](https://img.shields.io/badge/Version-1.8.4-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 2.3.1](https://img.shields.io/badge/AppVersion-2.3.1-informational?style=flat-square)

Grafana Tempo in MicroService mode

Expand Down Expand Up @@ -561,11 +561,12 @@ The memcached default args are removed and should be provided manually. The sett
| metricsGenerator.annotations | object | `{}` | Annotations for the metrics-generator StatefulSet |
| metricsGenerator.appProtocol | object | `{"grpc":null}` | Adds the appProtocol field to the metricsGenerator service. This allows metricsGenerator to work with istio protocol selection. |
| metricsGenerator.appProtocol.grpc | string | `nil` | Set the optional grpc service protocol. Ex: "grpc", "http2" or "https" |
| metricsGenerator.config | object | `{"metrics_ingestion_time_range_slack":"30s","processor":{"service_graphs":{"dimensions":[],"histogram_buckets":[0.1,0.2,0.4,0.8,1.6,3.2,6.4,12.8],"max_items":10000,"wait":"10s","workers":10},"span_metrics":{"dimensions":[],"histogram_buckets":[0.002,0.004,0.008,0.016,0.032,0.064,0.128,0.256,0.512,1.02,2.05,4.1]}},"registry":{"collection_interval":"15s","external_labels":{},"stale_duration":"15m"},"storage":{"path":"/var/tempo/wal","remote_write":[],"remote_write_flush_deadline":"1m","wal":null}}` | More information on configuration: https://grafana.com/docs/tempo/latest/configuration/#metrics-generator |
| metricsGenerator.config | object | `{"metrics_ingestion_time_range_slack":"30s","processor":{"service_graphs":{"dimensions":[],"histogram_buckets":[0.1,0.2,0.4,0.8,1.6,3.2,6.4,12.8],"max_items":10000,"wait":"10s","workers":10},"span_metrics":{"dimensions":[],"histogram_buckets":[0.002,0.004,0.008,0.016,0.032,0.064,0.128,0.256,0.512,1.02,2.05,4.1]}},"registry":{"collection_interval":"15s","external_labels":{},"stale_duration":"15m"},"storage":{"path":"/var/tempo/wal","remote_write":[],"remote_write_flush_deadline":"1m","wal":null},"traces_storage":{"path":"/var/tempo/traces"}}` | More information on configuration: https://grafana.com/docs/tempo/latest/configuration/#metrics-generator |
| metricsGenerator.config.processor.service_graphs | object | `{"dimensions":[],"histogram_buckets":[0.1,0.2,0.4,0.8,1.6,3.2,6.4,12.8],"max_items":10000,"wait":"10s","workers":10}` | For processors to be enabled and generate metrics, pass the names of the processors to overrides.metrics_generator_processors value like [service-graphs, span-metrics] |
| metricsGenerator.config.processor.service_graphs.dimensions | list | `[]` | resource and span attributes and are added to the metrics if present. |
| metricsGenerator.config.processor.span_metrics.dimensions | list | `[]` | Dimensions are searched for in the resource and span attributes and are added to the metrics if present. |
| metricsGenerator.config.storage.remote_write | list | `[]` | https://prometheus.io/docs/prometheus/latest/configuration/configuration/#remote_write |
| metricsGenerator.config.traces_storage | object | `{"path":"/var/tempo/traces"}` | Used by the local blocks processor to store a wal for traces. |
| metricsGenerator.enabled | bool | `false` | Specifies whether a metrics-generator should be deployed |
| metricsGenerator.extraArgs | list | `[]` | Additional CLI args for the metrics-generator |
| metricsGenerator.extraEnv | list | `[]` | Environment variables to add to the metrics-generator pods |
Expand All @@ -577,7 +578,12 @@ The memcached default args are removed and should be provided manually. The sett
| metricsGenerator.image.registry | string | `nil` | The Docker registry for the metrics-generator image. Overrides `tempo.image.registry` |
| metricsGenerator.image.repository | string | `nil` | Docker image repository for the metrics-generator image. Overrides `tempo.image.repository` |
| metricsGenerator.image.tag | string | `nil` | Docker image tag for the metrics-generator image. Overrides `tempo.image.tag` |
| metricsGenerator.kind | string | `"Deployment"` | Kind of deployment [StatefulSet/Deployment] |
| metricsGenerator.nodeSelector | object | `{}` | Node selector for metrics-generator pods |
| metricsGenerator.persistence | object | `{"annotations":{},"enabled":false,"size":"10Gi","storageClass":null}` | Persistence configuration for metrics-generator |
| metricsGenerator.persistence.annotations | object | `{}` | Annotations for metrics generator PVCs |
| metricsGenerator.persistence.enabled | bool | `false` | Enable creating PVCs if you have kind set to StatefulSet. This disables using local disk or memory configured in walEmptyDir |
| metricsGenerator.persistence.storageClass | string | `nil` | Storage class to be used. If defined, storageClassName: <storageClass>. If set to "-", storageClassName: "", which disables dynamic provisioning. If empty or set to null, no storageClassName spec is set, choosing the default provisioner (gp2 on AWS, standard on GKE, AWS, and OpenStack). |
| metricsGenerator.podAnnotations | object | `{}` | Annotations for metrics-generator pods |
| metricsGenerator.podLabels | object | `{}` | Labels for metrics-generator pods |
| metricsGenerator.ports | list | `[{"name":"grpc","port":9095,"service":true},{"name":"http-memberlist","port":7946,"service":false},{"name":"http-metrics","port":3100,"service":true}]` | Default ports |
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{- if .Values.metricsGenerator.enabled }}
{{- if and (.Values.metricsGenerator.enabled) (eq .Values.metricsGenerator.kind "Deployment") }}
{{ $dict := dict "ctx" . "component" "metrics-generator" "memberlist" true }}
apiVersion: apps/v1
kind: Deployment
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
{{- if and (.Values.metricsGenerator.enabled) (eq .Values.metricsGenerator.kind "StatefulSet") }}
{{ $dict := dict "ctx" . "component" "metrics-generator" "memberlist" true }}
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: {{ include "tempo.resourceName" $dict }}
namespace: {{ .Release.Namespace }}
labels:
{{- include "tempo.labels" $dict | nindent 4 }}
{{- with .Values.metricsGenerator.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
minReadySeconds: 10
replicas: {{ .Values.metricsGenerator.replicas }}
revisionHistoryLimit: 10
podManagementPolicy: Parallel
updateStrategy:
rollingUpdate:
partition: 0
selector:
matchLabels:
{{- include "tempo.selectorLabels" $dict | nindent 6 }}
template:
metadata:
labels:
{{- include "tempo.podLabels" $dict | nindent 8 }}
{{- with .Values.tempo.podLabels }}
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.metricsGenerator.podLabels }}
{{- toYaml . | nindent 8 }}
{{- end }}
annotations:
checksum/config: {{ include (print $.Template.BasePath "/configmap-tempo.yaml") . | sha256sum }}
{{- with .Values.tempo.podAnnotations }}
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.metricsGenerator.podAnnotations }}
{{- toYaml . | nindent 8 }}
{{- end }}
spec:
{{- if or (.Values.metricsGenerator.priorityClassName) (.Values.global.priorityClassName) }}
priorityClassName: {{ default .Values.metricsGenerator.priorityClassName .Values.global.priorityClassName }}
{{- end }}
serviceAccountName: {{ include "tempo.serviceAccountName" . }}
{{- with .Values.tempo.podSecurityContext }}
securityContext:
{{- toYaml . | nindent 8 }}
{{- end }}
enableServiceLinks: false
{{- include "tempo.metricsGeneratorImagePullSecrets" . | nindent 6 -}}
{{- with .Values.metricsGenerator.hostAliases }}
hostAliases:
{{- toYaml . | nindent 8 }}
{{- end }}
containers:
- args:
- -target=metrics-generator
- -config.file=/conf/tempo.yaml
- -mem-ballast-size-mbs=1024
{{- with .Values.metricsGenerator.extraArgs }}
{{- toYaml . | nindent 12 }}
{{- end }}
image: {{ include "tempo.imageReference" $dict }}
imagePullPolicy: {{ .Values.tempo.image.pullPolicy }}
name: metrics-generator
ports:
{{- range .Values.metricsGenerator.ports }}
- name: {{ .name | quote }}
containerPort: {{ .port }}
{{- end }}
{{- with .Values.metricsGenerator.extraEnv }}
env:
{{- toYaml . | nindent 12 }}
{{- end }}
{{- with .Values.metricsGenerator.extraEnvFrom }}
envFrom:
{{- toYaml . | nindent 12 }}
{{- end }}
readinessProbe:
{{- toYaml .Values.tempo.readinessProbe | nindent 12 }}
resources:
{{- toYaml .Values.metricsGenerator.resources | nindent 12 }}
{{- with .Values.tempo.securityContext }}
securityContext:
{{- toYaml . | nindent 12 }}
{{- end }}
volumeMounts:
- mountPath: /conf
name: config
- mountPath: /runtime-config
name: runtime-config
- mountPath: /var/tempo
name: wal
{{- if .Values.enterprise.enabled }}
- name: license
mountPath: /license
{{- end }}
{{- with .Values.metricsGenerator.extraVolumeMounts }}
{{- toYaml . | nindent 12 }}
{{- end }}
terminationGracePeriodSeconds: {{ .Values.metricsGenerator.terminationGracePeriodSeconds }}
{{- if semverCompare ">= 1.19-0" .Capabilities.KubeVersion.Version }}
{{- with .Values.metricsGenerator.topologySpreadConstraints }}
topologySpreadConstraints:
{{- tpl . $ | nindent 8 }}
{{- end }}
{{- end }}
{{- with .Values.metricsGenerator.affinity }}
affinity:
{{- tpl . $ | nindent 8 }}
{{- end }}
{{- with .Values.metricsGenerator.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.metricsGenerator.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
volumes:
- name: config
{{- include "tempo.configVolume" . | nindent 10 }}
- name: runtime-config
{{- include "tempo.runtimeVolume" . | nindent 10 }}
{{- if .Values.enterprise.enabled }}
- name: license
secret:
secretName: {{ tpl .Values.license.secretName . }}
{{- end }}
{{- with .Values.metricsGenerator.extraVolumes }}
{{- toYaml . | nindent 8 }}
{{- end }}
{{- if not .Values.metricsGenerator.persistence.enabled }}
- name: wal
emptyDir: {{- toYaml .Values.metricsGenerator.walEmptyDir | nindent 12 }}
{{- else }}
volumeClaimTemplates:
- metadata:
{{- with .Values.metricsGenerator.persistence.annotations }}
annotations:
{{- toYaml . | nindent 10 }}
{{- end }}
name: wal
spec:
accessModes:
- ReadWriteOnce
{{- with .Values.metricsGenerator.persistence.storageClass }}
storageClassName: {{ if (eq "-" .) }}""{{ else }}{{ . }}{{ end }}
{{- end }}
resources:
requests:
storage: {{ .Values.metricsGenerator.persistence.size | quote }}
{{- end }}
{{- end }}
20 changes: 20 additions & 0 deletions charts/tempo-distributed/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,8 @@ ingester:
metricsGenerator:
# -- Specifies whether a metrics-generator should be deployed
enabled: false
# -- Kind of deployment [StatefulSet/Deployment]
kind: Deployment
# -- Annotations for the metrics-generator StatefulSet
annotations: {}
# -- Number of replicas for the metrics-generator
Expand Down Expand Up @@ -293,6 +295,19 @@ metricsGenerator:
nodeSelector: {}
# -- Tolerations for metrics-generator pods
tolerations: []
# -- Persistence configuration for metrics-generator
persistence:
# -- Enable creating PVCs if you have kind set to StatefulSet. This disables using local disk or memory configured in walEmptyDir
enabled: false
size: 10Gi
# -- Storage class to be used.
# If defined, storageClassName: <storageClass>.
# If set to "-", storageClassName: "", which disables dynamic provisioning.
# If empty or set to null, no storageClassName spec is
# set, choosing the default provisioner (gp2 on AWS, standard on GKE, AWS, and OpenStack).
storageClass: null
# -- Annotations for metrics generator PVCs
annotations: {}
# -- The EmptyDir location where the /var/tempo will be mounted on. Defaults to local disk, can be set to memory.
walEmptyDir: {}
## Here shows how to configure 1Gi memory as emptyDir.
Expand Down Expand Up @@ -342,6 +357,9 @@ metricsGenerator:
# -- A list of remote write endpoints.
# -- https://prometheus.io/docs/prometheus/latest/configuration/configuration/#remote_write
remote_write: []
# -- Used by the local blocks processor to store a wal for traces.
traces_storage:
path: /var/tempo/traces
metrics_ingestion_time_range_slack: 30s
service:
# -- Annotations for Metrics Generator service
Expand Down Expand Up @@ -1100,6 +1118,8 @@ config: |
{{- toYaml .Values.metricsGenerator.config.processor | nindent 6 }}
storage:
{{- toYaml .Values.metricsGenerator.config.storage | nindent 6 }}
traces_storage:
{{- toYaml .Values.metricsGenerator.config.traces_storage | nindent 6 }}
registry:
{{- toYaml .Values.metricsGenerator.config.registry | nindent 6 }}
metrics_ingestion_time_range_slack: {{ .Values.metricsGenerator.config.metrics_ingestion_time_range_slack }}
Expand Down

0 comments on commit e880312

Please sign in to comment.