[Feature Request] Improve alerting

[timbastin]

Is your feature request related to a problem? Please describe

I'm really frustrated with opensearch alerting. We are trying to collect logs and kubernetes events in a kubernetes cluster. Send them to opensearch and create an alert rule, if an error event happens. I want to display this error event in a micrsoft teams channel.

Appearantly the combination of otel and opensearch and mustache makes this sheer impossible.

Using the Open Telemetry Kubernetes Events receiver OpenSearch gets logs in a nested structure.

It looks like this:

{
  "body": "xxx",
  "resource": {
    "attributes": {
      "k8s@namespace@name": "test-namespace"
    }
  }
}

OpenSearch now automatically flattens this structure before storing it.
If I am querying, everything works fine and I get the flattened structure back like this:

{
  "body": "xxx",
  "resource.attributes.k8s@namespace@name": "test-namespace"
}

Now I want to send a message to microsoft teams, which includes the body of the message and the namespace name to get more context. This seems to be impossible for me, since I am not able to access properties containing a "." in a mustache template.

Is there any cloud-native way to fix this issue? Creating a pipeline inside the ui and afterwards updating the deployment again seems like pretty manual solution. We are using opensearch using the official helm chart.

Describe the solution you'd like

I just want to access a property with a "dot" in it. This should be possible. I never decided to explicitly flatten the object. For me it feels like opensearch just decides to flatten the nested object and thus make it impossible to create nice contexualized alerts for it.

Related component

Other

Describe alternatives you've considered

No response

Additional context

No response