Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(cosmic-swingset): Split inbound queue length metrics by queue na…
…me (#10904) Fixes #10900 ## Description The first commit is pure refactoring. After that, kernel-stat.js is updated to document QUEUE_METRICS ({length,add,remove} triples sharing a common "cosmic_swingset_inbound_queue" prefix) and `makeInboundQueueMetrics` is updated to be an internal detail of that file (i.e., rather than constructing `inboundQueueMetrics`, launch-chain.js passes in initial queue lengths to `exportKernelStats` and extracts it from the response). This PR changes Prometheus `GET /metrics` output to add "queue" dimensional labels corresponding with the containing ~"forced"/"high-priority"/"queued" [CrankerPhase](https://github.com/Agoric/agoric-sdk/blob/f047c93df3ba5871cd9f576782f133e6d2bd21ff/packages/cosmic-swingset/src/launch-chain.js#L91)~ ("forced"/"priority"/"inbound" as of this work): ```diff # HELP cosmic_swingset_inbound_queue_length inbound queue length # TYPE cosmic_swingset_inbound_queue_length gauge -cosmic_swingset_inbound_queue_length 0 1738018661949 +cosmic_swingset_inbound_queue_length{queue="forced"} 0 1738018661949 +cosmic_swingset_inbound_queue_length{queue="priority"} 0 1738018661949 +cosmic_swingset_inbound_queue_length{queue="inbound"} 0 1738018661949 # HELP cosmic_swingset_inbound_queue_add inbound queue increments # TYPE cosmic_swingset_inbound_queue_add counter -cosmic_swingset_inbound_queue_add 0 1738018661949 +cosmic_swingset_inbound_queue_add{queue="forced"} 0 1738018661949 +cosmic_swingset_inbound_queue_add{queue="priority"} 0 1738018661949 +cosmic_swingset_inbound_queue_add{queue="inbound"} 0 1738018661949 # HELP cosmic_swingset_inbound_queue_remove inbound queue decrements # TYPE cosmic_swingset_inbound_queue_remove counter -cosmic_swingset_inbound_queue_remove 0 1738018661949 +cosmic_swingset_inbound_queue_remove{queue="forced"} 0 1738018661949 +cosmic_swingset_inbound_queue_remove{queue="priority"} 0 1738018661949 +cosmic_swingset_inbound_queue_remove{queue="inbound"} 0 1738018661949 ``` And slog entries to capture those same dimensions in object property name suffixes: ```diff { "type": "cosmic-swingset-end-block-finish", "blockHeight": 21, "blockTime": 1738022131, "inboundQueueStats": { + "cosmic_swingset_inbound_queue_length_forced": 0, + "cosmic_swingset_inbound_queue_length_priority": 0, + "cosmic_swingset_inbound_queue_length_inbound": 0, "cosmic_swingset_inbound_queue_length": 0, + "cosmic_swingset_inbound_queue_add_forced": 4, + "cosmic_swingset_inbound_queue_add_priority": 0, + "cosmic_swingset_inbound_queue_add_inbound": 0, "cosmic_swingset_inbound_queue_add": 4, + "cosmic_swingset_inbound_queue_remove_forced": 4, + "cosmic_swingset_inbound_queue_remove_priority": 0, + "cosmic_swingset_inbound_queue_remove_inbound": 0, "cosmic_swingset_inbound_queue_remove": 4 }, "time": 1738022137.072313, "monotime": 733.9035001497268 } ``` ### Security Considerations None known. ### Scaling Considerations This does increase the size of cosmic-swingset-begin-block and cosmic-swingset-end-block-finish slog entries, but I don't think to an extent that should dissuade us from including the detail. ### Documentation Considerations This introduces implicit documentation of cosmic-swingset metrics. ### Testing Considerations Expected output was confirmed manually (see the diffs above), but automated verification of expected instrument names, but automated verification of expected metric names would be better. ### Upgrade Considerations I believe that existing consumers of cosmic_swingset_inbound_queue_{length,add,remove} data should tolerate the new dimensionality and naïvely sum over all labels, giving the same data as before. Verification in live networks should cover that in addition to the above slogfile/Prometheus scrape[^1] checks. [^1]: as configured by environment variable `OTEL_EXPORTER_PROMETHEUS_PORT`.
- Loading branch information