Plugin restarted with panic debug log

[dkhwangbo]

Sometimes plugin shutdown with this debug log

logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.773188679+09:00 level=debug msg="panic: interface conversion: interface {} is float64, not string"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.773232026+09:00 level=debug msg=
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.773248611+09:00 level=debug msg="goroutine 101 [running]:"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.773270958+09:00 level=debug msg="main.(*druidDatasource).prepareResponse(0xba7280?, 0xc000566ac0, 0x1?)"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.773286979+09:00 level=debug msg="\t/workspace/pkg/druid.go:786 +0x1a3c"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.773518172+09:00 level=debug msg="main.(*druidDatasource).query(0xc0003a2240?, {{0x1180c88, 0x1}, {0x0, 0x0}, 0x1e6, 0x12a05f200, {{0x27fea5c0, 0xedb6bccbd, 0x11c29a0}, ...}, ...}, ...)"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.773673296+09:00 level=debug msg="\t/workspace/pkg/druid.go:307 +0x2e5"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.773713541+09:00 level=debug msg="main.(*druidDatasource).QueryData(0xc000285600, {0xba4880?, 0x0?}, 0xc00053e5f0)"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.773745586+09:00 level=debug msg="\t/workspace/pkg/druid.go:277 +0x20c"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.773762915+09:00 level=debug msg="github.com/grafana/grafana-plugin-sdk-go/backend.(*dataSDKAdapter).QueryData(0xc000285660, {0xd08690, 0xc000491650}, 0x0?)"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.773781714+09:00 level=debug msg="\t/root/go/pkg/mod/github.com/grafana/[email protected]/backend/data_adapter.go:21 +0x4d"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.773798363+09:00 level=debug msg="github.com/grafana/grafana-plugin-sdk-go/backend/grpcplugin.(*dataGRPCServer).QueryData(0xc0003c5990?, {0xd08690?, 0xc000491650?}, 0xc0004916e0?)"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.77382194+09:00 level=debug msg="\t/root/go/pkg/mod/github.com/grafana/[email protected]/backend/grpcplugin/grpc_data.go:47 +0x2b"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.773846176+09:00 level=debug msg="github.com/grafana/grafana-plugin-sdk-go/genproto/pluginv2._Data_QueryData_Handler.func1({0xd08690, 0xc000491650}, {0xb91480?, 0xc00053e460})"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.773870981+09:00 level=debug msg="\t/root/go/pkg/mod/github.com/grafana/[email protected]/genproto/pluginv2/backend_grpc.pb.go:191 +0x78"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.773995869+09:00 level=debug msg="github.com/grpc-ecosystem/go-grpc-prometheus.(*ServerMetrics).UnaryServerInterceptor.func1({0xd08690, 0xc000491650}, {0xb91480, 0xc00053e460}, 0x7fa0d06a0eb8?, 0xc000179740)"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.774038714+09:00 level=debug msg="\t/root/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/server_metrics.go:107 +0x87"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.77406597+09:00 level=debug msg="github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0xd08690?, 0xc000491650?}, {0xb91480?, 0xc00053e460?})"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.774094856+09:00 level=debug msg="\t/root/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/chain.go:25 +0x3a"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.774242169+09:00 level=debug msg="github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1({0xd08690, 0xc000491650}, {0xb91480, 0xc00053e460}, 0xc000106ae0?, 0xb0eaa0?)"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.774272839+09:00 level=debug msg="\t/root/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/chain.go:34 +0xbf"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.774398419+09:00 level=debug msg="github.com/grafana/grafana-plugin-sdk-go/genproto/pluginv2._Data_QueryData_Handler({0xaf9500?, 0xc000285920}, {0xd08690, 0xc000491650}, 0xc0004f4a20, 0xc0002a15c0)"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.774428602+09:00 level=debug msg="\t/root/go/pkg/mod/github.com/grafana/[email protected]/genproto/pluginv2/backend_grpc.pb.go:193 +0x138"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.774567956+09:00 level=debug msg="google.golang.org/grpc.(*Server).processUnaryRPC(0xc0002bf6c0, {0xd0c2a0, 0xc0003a0000}, 0xc0005339e0, 0xc0002a1e00, 0x11ab250, 0x0)"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.774597614+09:00 level=debug msg="\t/root/go/pkg/mod/google.golang.org/[email protected]/server.go:1279 +0xccf"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.774663131+09:00 level=debug msg="google.golang.org/grpc.(*Server).handleStream(0xc0002bf6c0, {0xd0c2a0, 0xc0003a0000}, 0xc0005339e0, 0x0)"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.774809747+09:00 level=debug msg="\t/root/go/pkg/mod/google.golang.org/[email protected]/server.go:1608 +0xa1b"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.774841898+09:00 level=debug msg=google.golang.org/grpc.(*Server).serveStreams.func1.2()
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.77486451+09:00 level=debug msg="\t/root/go/pkg/mod/google.golang.org/[email protected]/server.go:923 +0x98"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.77489583+09:00 level=debug msg="created by google.golang.org/grpc.(*Server).serveStreams.func1"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.774914002+09:00 level=debug msg="\t/root/go/pkg/mod/google.golang.org/[email protected]/server.go:921 +0x28a"
logger=plugin.grafadruid-druid-datasource t=2023-02-01T13:00:14.776924398+09:00 level=debug msg="plugin process exited" path=/var/lib/grafana/plugins/grafadruid-druid-datasource/grafadruid-druid-datasource_linux_amd64 pid=632651 error="exit status 2"

I'm using

• Grafana OSS v9.3.6
• grafadruid-druid-datasource v1.4.1
• Druid 0.21.0

[jbguerraz]

Hello @dkhwangbo
Thank you for the information!
Did you find a way to reproduce ? that would be super useful to debug it
There miss info logs from https://github.com/grafadruid/druid-grafana/blob/master/pkg/druid.go#L310

[jbguerraz]

Hello @dkhwangbo
Based on your debug log I bet you're not using 1.4.1 but instead an older version (line numbers doesn't match the methods in 1.4.1 but match in an old version).
Your problem is weird to me. The code run a rather simple algorithm to detect the data type since Druid returns mostly strings and the plugin have to tell grafana what data types the values are: to do so we check 5 column values and try to convert to detect what type it is. At the end we keep only the data type that succeed to convert the most for those 5 values.
In your case it says it's string but then when we want to cast to string it fails sayin' it's a float.
I'd like to get your results to understand the failure in type detection (a column is supposed to use only a single data type IMHO).
It's also the first time we hear about it so it must be an edge case.

[jbguerraz]

any news @dkhwangbo ?