forked from apache/solr
-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introduce prometheus metrics in Solr for Solr Core registry #1
Closed
Closed
Changes from all commits
Commits
Show all changes
25 commits
Select commit
Hold shift + click to select a range
3c3095a
Init commit for dropwizard to prometheus
749227d
Add request times
c58d2da
Refactor Prometheus metrics into utils
936aaeb
Create base SolrPrometheusMetrics class
428fe2a
Add new prometheus metric types
b418f68
Convert TLOG and Cache to Prometheus
178ee25
Add highlighter conversion
bd5d699
Convert INDEX metrics
adee592
Refactor handler conversion logic
a88af56
Create the SolrCoreMetric abstract class
ddfad4a
Fix broken Gauges not registering metrics
c4cc086
Remove comments
f3f6545
Refactor SolrCoreMetric categorization objects
f2637fe
Get core name from Solr cloud mode
5083d56
Refactor registering metric types
c4ccbe0
Introduce cloud mode metrics
7d11332
gradlew tidy
a3b9d19
Introduce timers gauges
b04bd14
Don't fail on entirely exporting a broken metric
771d823
Refactor function placements
fecd76d
Remove unused prometheus dependencies
f9b6d28
Move core metrics classes to core package
c6101f4
Add license agreement at top of new files
d3dc585
Update licenses
d00b249
Remove functions that break tests
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
93 changes: 93 additions & 0 deletions
93
solr/core/src/java/org/apache/solr/metrics/prometheus/SolrPrometheusCoreRegistry.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| /* | ||
| * Licensed to the Apache Software Foundation (ASF) under one or more | ||
| * contributor license agreements. See the NOTICE file distributed with | ||
| * this work for additional information regarding copyright ownership. | ||
| * The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| * (the "License"); you may not use this file except in compliance with | ||
| * the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
| package org.apache.solr.metrics.prometheus; | ||
|
|
||
| import com.codahale.metrics.Metric; | ||
| import io.prometheus.metrics.model.registry.PrometheusRegistry; | ||
| import org.apache.solr.metrics.prometheus.core.SolrCoreCacheMetric; | ||
| import org.apache.solr.metrics.prometheus.core.SolrCoreHandlerMetric; | ||
| import org.apache.solr.metrics.prometheus.core.SolrCoreHighlighterMetric; | ||
| import org.apache.solr.metrics.prometheus.core.SolrCoreIndexMetric; | ||
| import org.apache.solr.metrics.prometheus.core.SolrCoreMetric; | ||
| import org.apache.solr.metrics.prometheus.core.SolrCoreNoOpMetric; | ||
| import org.apache.solr.metrics.prometheus.core.SolrCoreSearcherMetric; | ||
| import org.apache.solr.metrics.prometheus.core.SolrCoreTlogMetric; | ||
|
|
||
| public class SolrPrometheusCoreRegistry extends SolrPrometheusRegistry { | ||
| public final String coreName; | ||
| public final boolean cloudMode; | ||
| public static final String ADMIN = "ADMIN"; | ||
| public static final String QUERY = "QUERY"; | ||
| public static final String UPDATE = "UPDATE"; | ||
| public static final String REPLICATION = "REPLICATION"; | ||
| public static final String TLOG = "TLOG"; | ||
| public static final String CACHE = "CACHE"; | ||
| public static final String SEARCHER = "SEARCHER"; | ||
| public static final String HIGHLIGHTER = "HIGHLIGHTER"; | ||
| public static final String INDEX = "INDEX"; | ||
| public static final String CORE = "CORE"; | ||
|
|
||
| public SolrPrometheusCoreRegistry( | ||
| PrometheusRegistry prometheusRegistry, String coreName, boolean cloudMode) { | ||
| super(prometheusRegistry); | ||
| this.coreName = coreName; | ||
| this.cloudMode = cloudMode; | ||
| } | ||
|
|
||
| public void exportDropwizardMetric(String metricName, Metric dropwizardMetric) { | ||
| SolrCoreMetric solrCoreMetric = categorizeCoreMetric(dropwizardMetric, metricName); | ||
| solrCoreMetric.parseLabels().toPrometheus(this); | ||
| } | ||
|
|
||
| private SolrCoreMetric categorizeCoreMetric(Metric dropwizardMetric, String metricName) { | ||
| String metricCategory = metricName.split("\\.")[0]; | ||
| switch (metricCategory) { | ||
| case ADMIN: | ||
| case QUERY: | ||
| case UPDATE: | ||
| case REPLICATION: | ||
| { | ||
| return new SolrCoreHandlerMetric(dropwizardMetric, coreName, metricName, cloudMode); | ||
| } | ||
| case TLOG: | ||
| { | ||
| return new SolrCoreTlogMetric(dropwizardMetric, coreName, metricName, cloudMode); | ||
| } | ||
| case CACHE: | ||
| { | ||
| return new SolrCoreCacheMetric(dropwizardMetric, coreName, metricName, cloudMode); | ||
| } | ||
| case SEARCHER: | ||
| { | ||
| return new SolrCoreSearcherMetric(dropwizardMetric, coreName, metricName, cloudMode); | ||
| } | ||
| case HIGHLIGHTER: | ||
| { | ||
| return new SolrCoreHighlighterMetric(dropwizardMetric, coreName, metricName, cloudMode); | ||
| } | ||
| case INDEX: | ||
| { | ||
| return new SolrCoreIndexMetric(dropwizardMetric, coreName, metricName, cloudMode); | ||
| } | ||
| case CORE: | ||
| default: | ||
| { | ||
| return new SolrCoreNoOpMetric(dropwizardMetric, coreName, metricName, cloudMode); | ||
| } | ||
| } | ||
| } | ||
| } | ||
144 changes: 144 additions & 0 deletions
144
solr/core/src/java/org/apache/solr/metrics/prometheus/SolrPrometheusRegistry.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,144 @@ | ||
| /* | ||
| * Licensed to the Apache Software Foundation (ASF) under one or more | ||
| * contributor license agreements. See the NOTICE file distributed with | ||
| * this work for additional information regarding copyright ownership. | ||
| * The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| * (the "License"); you may not use this file except in compliance with | ||
| * the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
| package org.apache.solr.metrics.prometheus; | ||
|
|
||
| import com.codahale.metrics.Meter; | ||
| import com.codahale.metrics.Timer; | ||
| import io.prometheus.metrics.core.metrics.Counter; | ||
| import io.prometheus.metrics.core.metrics.Gauge; | ||
| import io.prometheus.metrics.model.registry.PrometheusRegistry; | ||
| import java.util.ArrayList; | ||
| import java.util.HashMap; | ||
| import java.util.Map; | ||
|
|
||
| public abstract class SolrPrometheusRegistry { | ||
| PrometheusRegistry prometheusRegistry; | ||
| String registryName; | ||
| private final Map<String, Counter> metricCounters; | ||
| private final Map<String, Gauge> metricGauges; | ||
|
|
||
| public SolrPrometheusRegistry(PrometheusRegistry prometheusRegistry) { | ||
| this.prometheusRegistry = prometheusRegistry; | ||
| this.metricCounters = new HashMap<>(); | ||
| this.metricGauges = new HashMap<>(); | ||
| } | ||
|
|
||
| public PrometheusRegistry getPrometheusRegistry() { | ||
| return prometheusRegistry; | ||
| } | ||
|
|
||
| public String getRegistryName() { | ||
| return registryName; | ||
| } | ||
|
|
||
| public void exportMeter( | ||
| Meter dropwizardMetric, String prometheusMetricName, Map<String, String> labelsMap) { | ||
| if (!metricCounters.containsKey(prometheusMetricName)) { | ||
| ArrayList<String> labels = new ArrayList<>(labelsMap.keySet()); | ||
| registerCounter(prometheusMetricName, labels.toArray(String[]::new)); | ||
| } | ||
| ArrayList<String> labelValues = new ArrayList<>(labelsMap.values()); | ||
| getMetricCounter(prometheusMetricName) | ||
| .labelValues(labelValues.toArray(String[]::new)) | ||
| .inc(dropwizardMetric.getCount()); | ||
| } | ||
|
|
||
| public void exportCounter( | ||
| com.codahale.metrics.Counter dropwizardMetric, | ||
| String prometheusMetricName, | ||
| Map<String, String> labelsMap) { | ||
| if (!metricCounters.containsKey(prometheusMetricName)) { | ||
| ArrayList<String> labels = new ArrayList<>(labelsMap.keySet()); | ||
| registerCounter(prometheusMetricName, labels.toArray(String[]::new)); | ||
| } | ||
| ArrayList<String> labelValues = new ArrayList<>(labelsMap.values()); | ||
| getMetricCounter(prometheusMetricName) | ||
| .labelValues(labelValues.toArray(String[]::new)) | ||
| .inc(dropwizardMetric.getCount()); | ||
| } | ||
|
|
||
| public void exportTimer( | ||
| Timer dropwizardMetric, String prometheusMetricName, Map<String, String> labelsMap) { | ||
| if (!metricCounters.containsKey(prometheusMetricName)) { | ||
| ArrayList<String> labels = new ArrayList<>(labelsMap.keySet()); | ||
| registerGauge(prometheusMetricName, labels.toArray(String[]::new)); | ||
| } | ||
| ArrayList<String> labelValues = new ArrayList<>(labelsMap.values()); | ||
| getMetricGauge(prometheusMetricName) | ||
| .labelValues(labelValues.toArray(String[]::new)) | ||
| .set(dropwizardMetric.getMeanRate()); | ||
| } | ||
|
|
||
| public void exportGauge( | ||
| com.codahale.metrics.Gauge<?> dropwizardMetricRaw, | ||
| String prometheusMetricName, | ||
| Map<String, String> labelsMap) { | ||
| Object dropwizardMetric = (dropwizardMetricRaw).getValue(); | ||
| if (!metricGauges.containsKey(prometheusMetricName)) { | ||
| ArrayList<String> labels = new ArrayList<>(labelsMap.keySet()); | ||
| if (dropwizardMetric instanceof HashMap) { | ||
| labels.add("item"); | ||
| } | ||
| registerGauge(prometheusMetricName, labels.toArray(String[]::new)); | ||
| } | ||
| ArrayList<String> labelValues = new ArrayList<>(labelsMap.values()); | ||
| String[] labels = labelValues.toArray(String[]::new); | ||
| if (dropwizardMetric instanceof Number) { | ||
| getMetricGauge(prometheusMetricName) | ||
| .labelValues(labels) | ||
| .set(((Number) dropwizardMetric).doubleValue()); | ||
| } else if (dropwizardMetric instanceof HashMap) { | ||
| HashMap<?, ?> itemsMap = (HashMap<?, ?>) dropwizardMetric; | ||
| for (Object item : itemsMap.keySet()) { | ||
| if (itemsMap.get(item) instanceof Number) { | ||
| String[] newLabels = new String[labels.length + 1]; | ||
| System.arraycopy(labels, 0, newLabels, 0, labels.length); | ||
| newLabels[labels.length] = (String) item; | ||
| getMetricGauge(prometheusMetricName) | ||
| .labelValues(newLabels) | ||
| .set(((Number) itemsMap.get(item)).doubleValue()); | ||
| } | ||
| } | ||
| } | ||
| } | ||
|
|
||
| private Counter getMetricCounter(String metricName) { | ||
| return metricCounters.get(metricName); | ||
| } | ||
|
|
||
| private Gauge getMetricGauge(String metricName) { | ||
| return metricGauges.get(metricName); | ||
| } | ||
|
|
||
| private void registerCounter(String metricName, String... labelNames) { | ||
| Counter counter = | ||
| io.prometheus.metrics.core.metrics.Counter.builder() | ||
| .name(metricName) | ||
| .labelNames(labelNames) | ||
| .register(prometheusRegistry); | ||
| metricCounters.put(metricName, counter); | ||
| } | ||
|
|
||
| private void registerGauge(String metricName, String... labelNames) { | ||
| Gauge gauge = | ||
| io.prometheus.metrics.core.metrics.Gauge.builder() | ||
| .name(metricName) | ||
| .labelNames(labelNames) | ||
| .register(prometheusRegistry); | ||
| metricGauges.put(metricName, gauge); | ||
| } | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just curious on your thoughts about why the other cases are empty instead of removing them?