Special characters in prometheus job name

[erachenko23]

Hello, sometimes we recieve error_code:400 on some alarms from prometheus alert manager:

Example:

2021-03-02 12:38:17,093 [DEBUG] com.github.mikesafonov.prometheus.alerts.starter.AlertEndpoint - Notification: AlertManagerNotification(version=4, receiver=alertmananger-bot, status=firing, groupLabels={instance=10.246.4.23:30657, severity=critical}, commonLabels={alertname=InstanceDown, environment=xxx.xxx.xxx.xx, instance=10.246.4.23:30657, job=app_prod, severity=critical}, commonAnnotations={description=Instance 10.246.4.23:30657 app_prod has been down for more than 5 minutes, summary=Instance 10.246.4.23:30657 app_prod has been down for more than 5 minutes, title=InstanceDown}, externalURL=http://monitor.xxx.xxx.xx:9093, alerts=[Alert(labels={alertname=InstanceDown, environment=monitor.xxx.xxx.xx, instance=10.246.4.23:30657, job=app_prod, severity=critical}, annotations={description=Instance 10.246.4.23:30657 app_prod has been down for more than 5 minutes, summary=Instance 10.246.4.23:30657 app_prod has been down for more than 5 minutes, title=InstanceDown}, startsAt=2021-02-26T08:11:44.318691705Z[UTC], endsAt=0001-01-01T00:00Z[UTC], generatorURL=http://xxx.xxx.xxx.xx:9090/graph?g0.expr=up+%3D%3D+0&g0.tab=1, status=firing)])
2021-03-02 12:38:17,093 [DEBUG] com.github.mikesafonov.prometheus.telegram.service.TelegramNotificationService - Telegram message:⚠ InstanceDown

2021-03-02 12:38:17,156 [ERROR] org.apache.catalina.core.ContainerBase.[Tomcat].[localhost].[/].[dispatcherServlet] - Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.web.client.HttpClientErrorException$BadRequest: 400 Bad Request: [{"ok":false,"error_code":400,"description":"Bad Request: can't parse entities: Can't find end of the entity starting at byte offset 48"}]] with root cause
org.springframework.web.client.HttpClientErrorException$BadRequest: 400 Bad Request: [{"ok":false,"error_code":400,"description":"Bad Request: can't parse entities: Can't find end of the entity starting at byte offset 48"}]

It happened because some job contains character '_' in name. When we change the character to '-' in the job name, the problem is solved.