-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Kafka: add hook to log connection error reasons #541
Conversation
Unpleasant discovery: the newly added test is currently flaky because of franz-go implicitly retrying to reconnect at least once, in handleReq Test failure here. I think we should disable the retry to make the test consistent but I am unsure how to pass that option to the client 🤔 |
Signed-off-by: inge4pres <[email protected]>
Signed-off-by: inge4pres <[email protected]>
Signed-off-by: inge4pres <[email protected]>
Signed-off-by: inge4pres <[email protected]>
Co-authored-by: Edoardo Tenani <[email protected]> Signed-off-by: inge4pres <[email protected]>
Signed-off-by: inge4pres <[email protected]>
85faa63
to
6194881
Compare
Not seeing an option to control the connection retry, I ended up changing the assertion: we now validate that there is either a single or 2 log lines containing the error |
Reason for this PR
The current
metricsHook
struct that collects metrics based on Kafka hooks is not equipped to also report details of why connection errors are caused.Details
We provide a new struct that implements a (single, for now) kgo Hook dedicated to logging errors during the broker connection phase.
The struct can be iterated in the future to implement other monitoring hooks, providing logging utilities where the metrics would have too high cardinality.
Additional context
Currently, the proposed design adds the new logger hook behind the
CommonConfig.DisableTelemetry
flag (similarly to the metrics hook).This can be improved and it is not a perfect solution, but it has to be done this way until a bigger refactor is done to expose the
CommonConfig.hook
fieldin some way.The more idiomatic, but also more expensive, solution would be to allow exporting franz-go logs into the Logger.
This is mentioned in #540