OpenTelemetry tracing

[sideninja]

Integrate open telemetry tracing inside the GraphQL and using prometheus as the exporter. At this point we should just trace requests/responses no need to yet dive deeper in the codebase with traces as that will require refactoring of the functions to include passing of context (which we should do at later point anyway). Use Go opentelemtry library https://pkg.go.dev/github.com/rot1024/otelgqlgen
Some examples:
https://github.com/open-telemetry/opentelemetry-go/blob/main/example/prometheus/main.go

[DylanTinianov]

Needs SRE to enable Grafana Tracing Agent. Then we can simple update the telemetry config env var and re-deploy