Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Response success status validated incorrectly #4338

Open
slyapustin opened this issue Dec 5, 2024 · 11 comments · May be fixed by #4339
Open

Response success status validated incorrectly #4338

slyapustin opened this issue Dec 5, 2024 · 11 comments · May be fixed by #4339
Labels
bug Something isn't working

Comments

@slyapustin
Copy link

Describe your environment

OS: (e.g, Ubuntu)
Python version: (e.g., Python 3.8.10)
SDK version: (e.g., 1.25.0)
API version: (e.g., 1.25.0)

What happened?

There are some places, where success status of the request validated based on only 200 and 202 status code:

Steps to Reproduce

Receive the HTTP 204 succes status code

Expected Result

204 status to be treated as a success status

Actual Result

204 status code treated as an error

Additional context

Relevant issue (fixed) about the same behavior in another module: #3621

Would you like to implement a fix?

None

@slyapustin slyapustin added the bug Something isn't working label Dec 5, 2024
@codefromthecrypt
Copy link
Contributor

zipkin doesn't define its HTTP POST status as returning 204, rather 202 as the code already does.

https://github.com/openzipkin/zipkin-api/blob/master/zipkin-api.yaml#L83

What product is returning 204? Can you raise an issue with them to switch it?

@stfines-clgx
Copy link

I get it when using Arize Phoenix and Elastic Search. Maybe somebody could convince arize, but I doubt that one could convince elastic.

@codefromthecrypt
Copy link
Contributor

hmm the collector codebase returns correct HTTP status. I'll ask my colleagues at Elastic and also my friend at Arize

https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/70ab6e9b8ae96e4b7c59188672b1de0c93bb3e7d/receiver/zipkinreceiver/trace_receiver.go#L247

@stfines-clgx
Copy link

I'm getting mercilessly spammed with 204's hitting the Arize open-telemetry http endpoint when I do local development -
DEBUG:urllib3.connectionpool:http://localhost:6006 "POST /v1/traces HTTP/1.1" 204 0 ERROR:opentelemetry.exporter.otlp.proto.http.trace_exporter:Failed to export batch code: 204, reason:

@codefromthecrypt
Copy link
Contributor

zipkin's POST endpoint is /api/v1/spans, and that's what the collector correctly binds as well.

Seeing /v1/traces (OTLP) makes me think possibly there's a mismatch on exporter/receiver.

p.s. By default, zipkin itself doesn't bind the OTLP endpoint /v1/traces, though there is a different docker image that supports that recently

@stfines-clgx
Copy link

well, I'm using Arize Phoenix, not zipkin so I'm not sure where that comes in to play here.

@codefromthecrypt
Copy link
Contributor

@stfines-clgx so the issue as opened was a complaint about the code in zipkin exporter (not OTLP), and if it should accept HTTP 204 (which is not something defined by zipkin as it defines it uses 202).

I think your concern is related to 204 code, but not related to zipkin defining that. Rather it is about OTLP expecting 200, yet Arize Phoenix seems to be returning 204.

So, I think yes both scenarios are about endpoints returning 204 when something else is defined:

  • Zipkin /api/v1/spans -> 202
  • OTLP /v1/traces -> 200

@mikeldking
Copy link

Hey @stfines-clgx ! One of the maintainers of Phoenix. - @codefromthecrypt thanks for the heads up! Commenting to follow along. Feel free to tag me if you find it to be an issue with Phoenix!

@codefromthecrypt
Copy link
Contributor

raised this to follow up on the Phoenix side Arize-ai/phoenix#5963

@mikeldking
Copy link

fixed in Arize Phoenix. Thanks for flagging. https://github.com/Arize-ai/phoenix/releases/tag/arize-phoenix-v7.6.0

@codefromthecrypt
Copy link
Contributor

codefromthecrypt commented Jan 9, 2025

@stfines-clgx I did research into Elastic (due to the earlier note). As far as I can tell, there's no OTLP trace receiver code that can return a 204 (e.g apm server returns 200, and zipkin endpoints aren't present in elastic products either).

So, unless there's anything else, let's get back to this topic about the zipkin exporter.

@slyapustin can you double-check assumptions and mention the product you are exporting traces to? Basically, pick up from this comment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
4 participants