After upgrading the grafana to Version 11.3.0, not able to see the data from check-mk data source (2.2.0p35.cee)

[Prakashadapala]

After upgrading the grafana to Version 11.3.0, not able to see the data from check-mk data source (2.2.0p35.cee)

Grafana: Version 11.3.0
OS version: OEL 8.5
CheckMK Plugin: tribe-29-checkmk-datasource-3.3.0.zip

CMK version: 2.2.0p35.cee
OS version: OEL 9.4

Data source is working, reached version 2.2.0p35.cee of Checkmk

When i am trying to plot the graph able to user filters where i can see checkmk site name and all hostname and service details, but when i try to plot the graph no data is pulled.(Displaying message No data available either its any service (CPU, DISK, IO)

[BenediktSeidl]

can you check out the current beta release?
can you see any failed http requests in the network tab of the developer tools? what is the request or response of those requests?
do you see an error similar to this one? what's the exact wording?

[Prakashadapala]

@BenediktSeidl

I am not able to find any data in grafana even though i selected host details and predefine graph

here you can find logs on garafana server
logger=grafana-apiserver t=2024-11-14T02:54:42.615366242Z level=info msg="Adding GroupVersion featuretoggle.grafana.app v0alpha1 to ResourceManager"
logger=grafana-apiserver t=2024-11-14T02:54:42.616985943Z level=info msg="Adding GroupVersion iam.grafana.app v0alpha1 to ResourceManager"
logger=context userId=0 orgId=0 uname= t=2024-11-14T02:54:59.669991027Z level=info msg="Request Completed" method=GET path=/api/live/ws status=401 remote_addr=time_ms=0 duration=847.132µs size=105 referer= handler=/api/live/ws status_source=server
logger=context userId=0 orgId=0 uname= t=2024-11-14T02:55:13.406260346Z level=info msg="Request Completed" method=GET path=/ status=302 remote_addr=time_ms=0 duration=743.316µs size=47 referer= handler=/ status_source=server
logger=context userId=0 orgId=0 uname= t=2024-11-14T02:55:13.660951975Z level=info msg="Request Completed" method=GET path=/user/auth-tokens/rotate status=302 remote_addr=time_ms=5 duration=5.106325ms size=24 referer= handler=/user/auth-tokens/rotate status_source=server
logger=context userId=1 orgId=1 uname=admin t=2024-11-14T02:55:15.411007076Z level=info msg="Request Completed" method=GET path=/api/live/ws status=-1 remote_addr=time_ms=2 duration=2.287355ms size=0 referer= handler=/api/live/ws status_source=server
logger=context userId=1 orgId=1 uname=admin t=2024-11-14T02:55:18.651692019Z level=info msg="Request Completed" method=GET path=/api/live/ws status=-1 remote_addr=time_ms=2 duration=2.430408ms size=0 referer= handler=/api/live/ws status_source=server
logger=context userId=0 orgId=0 uname= t=2024-11-14T02:55:28.161356429Z level=info msg="Request Completed" method=GET path=/api/live/ws status=401 remote_addr=time_ms=0 duration=811.279µs size=105 referer= handler=/api/live/ws status_source=server
logger=context userId=1 orgId=1 uname=admin t=2024-11-14T02:55:31.359846109Z level=info msg="Request Completed" method=POST path=/api/datasources/proxy/uid/ee3mpjha7s6psa/rest/check_mk/api/1.0/objects/autocomplete/combined_graphs status=400 remote_addr=time_ms=381 duration=381.246827ms size=251 referer="http://usnc1grap1lvd:3000/dashboard/new?editPanel=1&from=now-6h&timezone=browser&to=now" handler=/api/datasources/proxy/uid/:uid/* status_source=server
logger=context userId=0 orgId=0 uname= t=2024-11-14T02:56:28.651742401Z level=info msg="Request Completed" method=GET path=/api/live/ws status=401 remote_addr=time_ms=0 duration=969.908µs size=105 referer= handler=/api/live/ws status_source=server
logger=infra.usagestats t=2024-11-14T02:56:33.166810945Z level=info msg="Usage stats are ready to report"
logger=context userId=0 orgId=0 uname= t=2024-11-14T02:57:29.447537304Z level=info msg="Request Completed" method=GET path=/api/live/ws status=401 remote_addr=time_ms=1 duration=1.161542ms size=105 referer= handler=/api/live/ws status_source=server
logger=context userId=0 orgId=0 uname= t=2024-11-14T02:58:29.936885381Z level=info msg="Request Completed" method=GET path=/api/live/ws status=401 remote_addr=time_ms=0 duration=901.452µs size=105 referer= handler=/api/live/ws status_source=server
logger=context userId=0 orgId=0 uname= t=2024-11-14T02:59:35.837784338Z level=info msg="Request Completed" method=GET path=/api/live/ws status=401 remote_addr=time_ms=1 duration=1.014114ms size=105 referer= handler=/api/live/ws status_source=server
logger=context userId=0 orgId=0 uname= t=2024-11-14T03:00:36.640113031Z level=info msg="Request Completed" method=GET path=/api/live/ws status=401 remote_addr=time_ms=0 duration=740.619µs size=105 referer= handler=/api/live/ws status_source=server

[Prakashadapala]

@BenediktSeidl Even tried with the mention beta version, same not able to fetch the data, i am able to get only hostdetails from checkmk data source, but not able to plot the resource utilization graphs

Here you can find the latest logs in grafana, post beta version.
logger=grafana.update.checker t=2024-11-14T03:05:09.120035073Z level=info msg="Update check succeeded" duration=130.106225ms
logger=grafana-apiserver t=2024-11-14T03:05:09.343780134Z level=info msg="Adding GroupVersion featuretoggle.grafana.app v0alpha1 to ResourceManager"
logger=grafana-apiserver t=2024-11-14T03:05:09.34550953Z level=info msg="Adding GroupVersion iam.grafana.app v0alpha1 to ResourceManager"
logger=grafana-apiserver t=2024-11-14T03:05:09.346220317Z level=info msg="Adding GroupVersion playlist.grafana.app v0alpha1 to ResourceManager"
logger=context userId=1 orgId=1 uname=admin t=2024-11-14T03:05:09.446183404Z level=info msg="Request Completed" method=GET path=/api/live/ws status=-1 remote_addr= time_ms=12 duration=12.577189ms size=0 referer= handler=/api/live/ws status_source=server
logger=context userId=1 orgId=1 uname=admin t=2024-11-14T03:05:10.00753904Z level=info msg="Request Completed" method=GET path=/api/live/ws status=-1 remote_addr= time_ms=2 duration=2.346153ms size=0 referer= handler=/api/live/ws status_source=server
logger=context userId=1 orgId=1 uname=admin t=2024-11-14T03:05:23.688943471Z level=info msg="Request Completed" method=GET path=/api/gnet/plugins/tribe-29-checkmk-datasource status=404 remote_addr= time_ms=109 duration=109.834553ms size=116 referer=http://usnc1grap1lvd:3000/connections/datasources/edit/ee3mpjha7s6psa handler=/api/gnet/* status_source=server
logger=context userId=1 orgId=1 uname=admin t=2024-11-14T03:05:37.090366063Z level=info msg="Request Completed" method=POST path=/api/datasources/proxy/uid/ee3mpjha7s6psa/rest/check_mk/api/1.0/objects/autocomplete/combined_graphs status=400 remote_addr= time_ms=372 duration=372.386581ms size=251 referer="http://usnc1grap1lvd:3000/explore?orgId=1&panes=%7B%22erj%22%3A%7B%22datasource%22%3A%22ee3mpjha7s6psa%22%2C%22queries%22%3A%5B%7B%22refId%22%3A%22A%22%2C%22datasource%22%3A%7B%22type%22%3A%22tribe-29-checkmk-datasource%22%2C%22uid%22%3A%22ee3mpjha7s6psa%22%7D%7D%5D%2C%22range%22%3A%7B%22from%22%3A%22now-1h%22%2C%22to%22%3A%22now%22%7D%7D%7D&schemaVersion=1" handler=/api/datasources/proxy/uid/:uid/* status_source=server
logger=context userId=0 orgId=0 uname= t=2024-11-14T03:05:40.540698488Z level=info msg="Request Completed" method=GET path=/api/live/ws status=401 remote_addr= time_ms=0 duration=726.371µs size=105 referer= handler=/api/live/ws status_source=server
logger=infra.usagestats t=2024-11-14T03:06:14.015700861Z level=info msg="Usage stats are ready to report"

[BenediktSeidl]

this looks suspicious:

method=POST path=/api/datasources/proxy/uid/ee3mpjha7s6psa/rest/check_mk/api/1.0/objects/autocomplete/combined_graphs status=400

you should see this request in the browser when you open the developer console and open the network tab. we need to know the exact request and response of checkmk in order to move forward.

what's very strange is, that you screenshot actually shows the legend, but no graph data, both are served in the same request, so checkmk sends some data. you only display the last 24 hours. can you show maybe the last year?

does you service or hostname contain some special characters? status code 400 could be a sign that some validation of the request parameter failed.

your screenshot only contains part of the query. can you show the whole query? are all fields filled out?

[Prakashadapala]

Here you can find the query details:
{
"request": {
"method": "POST",
"url": "api/datasources/proxy/uid/ee3vt8iejdt6oe/rest/check_mk/api/1.0/domain-types/metric/actions/filter/invoke",
"data": {
"filter": {
"host": {
"host": "depb3cijnkde2lvp"
}
},
"aggregation": "off",
"type": "predefined_graph",
"time_range": {
"start": "2024-11-14T01:18:56.000Z",
"end": "2024-11-14T07:18:56.000Z"
},
"graph_id": "cpu_load"
},
"hideFromInspector": false
},
"response": {
"time_range": {
"start": "2024-11-14T01:18:00+00:00",
"end": "2024-11-14T07:18:00+00:00"
},
"step": 60,
"metrics": [
{
"color": "#00d1ff",
"data_points": [
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null
],
"line_type": "line",
"title": "CPU load average of last minute"
},
{
"color": "#679bad",
"data_points": [
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null
],
"line_type": "line",
"title": "CPU load average of last 5 minutes"
},
{
"color": "#2c5766",
"data_points": [
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null
],
"line_type": "line",
"title": "CPU load average of last 15 minutes"
}
]
}
}

[BenediktSeidl]

this looks like there are no metrics recorded for your host. can you open your host in checkmk and see if there are metrics shown there?

[Prakashadapala]

I am able to see the data in Checkmk

[BenediktSeidl]

your screenshot does not show the graph of CPU load, just the services of the host. can you click on CPU load? do you see any graphs there?

[Prakashadapala]

Here it is:

[BenediktSeidl]

yep, this graph is empty. the yellow and red bar shows the configured warn and crit levels, they are not part of the data fetched by the grafana plugin.

this is what it should look like:

(my warn/crit is quite high, but the two lines are there:)

please ask in the forum on how to get the graph fixed, i'm not an expert on this area. maybe you can test some other graph to test that the communication with checkmk works?!

[Prakashadapala]

@BenediktSeidl Thanks for your time, i see post checkmk upgrade the graphs are not getting updated, let me fix that first and come back to grafana. Once again thanks for your help.