[NEW] OBSERVE command for enhanced observability in Valkey: Data Collection
#1167
Comments
|
I like the concepts and directionality here. I think it would be profitable to split this into two subsections. One section would be to get more specific on the events that would feed into the observability framework. A second section would focus on the representation and processing of those events. I mention this because there's quite a bit of overlap in the functionality of the second section and any implementation of a timestream processing module. In other words, can we get both timestream processing of the observability event stream (part 1) and more generic timestream data processing capabilities in the same development effort or conversely split the development effort into two parts that cooperate? |
|
@allenss-amazon Thank you for the insightful feedback! We’re definitely on the same page about maintaining flexibility for future enhancements, particularly around timestream processing and event-based observability. For now, our approach is to keep the implementation streamlined and tailored to Valkey’s current state of the codebase. By initially focusing on passing command execution details directly as the input to our observability pipeline, we’ll establish a foundation that can be then adapted for timestream events in the future. If and when we introduce events, we could replace or extend the input system for our pipeline to accommodate event-based data, allowing for similar processing while offering additional data input types. Keeping a direct dependency between command execution and the observability implementation will help us maintain a simple architecture and deliver a more focused solution in the short term. I’ve also included a simplified diagram of this first-approach implementation to illustrate the flow we envision. I’d love to hear your thoughts on whether this approach makes sense. What do you think? flowchart LR
A[User: AnyCommand] --> B("observePostCommand()")
subgraph valkey-server
subgraph "Command Execution: call()"
B
end
subgraph "observeUnitsProcess()"
B --> C("Filter()")
C --> D("Sample()")
D --> E("Partition()")
E --> F("Window()")
F --> G("Reduce()")
G --> H("Output()")
end
H --> I("Observe Results Store")
end
Thanks again for the input—we really appreciate the foresight around extensibility here! |
|
I think it's important to provide an architecture that's lightweight enough that it could be enabled nearly everywhere -- maybe even by default. That means that the data collection needs to be fast. I think it's important to get more specific about the kinds of data events you're going to rely on and how this data is generated. |
Absolutely. Performance is one of the primary goals. My initial approach is to check whether any observe functionality needs to do something after command executes. Currently, this creates an typedef struct observeUnit {
int command_id;
robj **argv;
size_t argv_len;
size_t response_size_bytes;
long long duration_microseconds;
} observeUnit;If the observe functionality is disabled or unconfigured, there’s no impact on command execution performance—it’s just a simple check of a boolean flag. If it’s enabled, I construct this unit on the stack and attempt to process it through the pipeline. (Note that the pipeline processing part hasn’t been implemented yet.)
At this early stage, the structs feel lightweight and execution should be 'fast', but I anticipate that 'data gathering' may need additional complexity. Currently, @allenss-amazon What do you think about my approach? Do you have any suggestions for enhancing 'data gathering' for observability? |
|
I'm skeptical of the "one size fits all" interface style here. For example, using the end of call as your insertion point will miss all client commands that block. But that clearly can be fixed by tapping into the unblocking machinery also. This also misses all non-command related activity -- cluster bus, evictions, etc. Which I think could also be quite interesting. I'd propose that we implement a mechanism to self-monitor the existing metrics in the core. For example, having a periodic task to execute an "info" command and collect the various values into samples to feed into the machinery could also be quite valuable. The automatic info scheme has a low enough overhead (you adjust the frequency of collection to match your CPU wallet ;-)) that it could reasonably be left on in any production environment. It also creates an incentive to increase instrumentation in the core. |
The primary goal is to design observability pipelines with enough flexibility to support a variety of input sources. In my initial implementation, I focused on integrating the first input source as 'command executions.' However, with the right design, we should be able to expand this model to support additional sources or even rework the data push mechanism to accommodate a new system based on event streams. Here’s a diagram illustrating this approach:
The concept is that multiple sources can feed into the Observe Units Processor, which will process them through the pipelines. Additionally, the Observe Units Processor should allow us to easily implement new input sources. (I may need some guidance on how best to structure this in the code.)
You’re right that the initial input source doesn’t cover several internal cluster activities (and blocking commands), but the design should allow us to extend the list of sources in future iterations. Structuring this properly may require some guidance, particularly to ensure compatibility with all potential Valkey input streams. Output from the INFO command is already accessible to the Clients, so I wonder—if we limited ourselves to just these results, would it be valuable enough to build out the entire observability pipeline? It’s feasible for users to set up a custom client to periodically fetch INFO data and compute time-series metrics. To bring real value to the observability pipeline, I aimed to start with something less/not accessible in the current Valkey feature set, while aligning with Google's 4 Golden Signals. Hence, I opted to implement this at the command level (though we could debate if this approach is ideal). What do you think? Are there any other arguments for limiting ourselves to the INFO results besides simplicity / incentives? |
|
@allenss-amazon said:
This is an excellent idea. @mwarzynski said:
@mwarzynski I like your approach to Allen's concept … IIRC, you want to split (a) the data collection (which can include filters for efficiency) and (b) the trigger from (c) what you term "Observe Units Processing". Then the "data collection" can come (i) command post processing like you're prototyping, or (ii) triggered by another command's execution including but not limited to the INFO command, or (iii) even timers for fully internal sampling. This frees us up to focus on multiple different parts of the implementation. We can discuss the programmable pipeline implementation while keeping the "data collection" part open for significant extension as we more use cases emerge. @mwarzynski said:
I think @allenss-amazon's idea is not to limit us to INFO but rather to make INFO command execution be able to feed the processing pipeline as a data source in addition to what you have already been working on. I gather from Allen's comment that the flexibility to be able to feed data (e.g. INFO or other commands) would motivate developers to feed more data into the observability pipeline and potentially have it "always on". Overall, I see the conceptual two stages "collection" and "programmable processing" as a powerful combination. Also note that pulling detailed command results (e.g. from INFO) out to the client is very expensive which might limit use cases. However, as Allen hints, having verbose output instead go into a server-side pipeline, especially with filters, could make it feasible to run "always on" observability. @allenss-amazon Please correct as appropriate … I don't want to go in a direction that you didn't intend. Overall, I think we have immediate use cases for memory sizing that require data collection from observePostCommand so I'd appreciate if we can start with that and add support for (x) INFO (among other commands) as input, (y) time/event/notifications triggers as a second step. Mostly, I just want to lock down the initial scope so that @mwarzynski can start work on the programmable pipeline part which is where we will discover how tricky this business is. |
|
Yes, multiple sources of data feeding the analysis engine. I believe a time-series processing module is in Valkey's future (likely with fidelity to the Redis time series module) and that this observability proposal should use that module as it's analysis engine rather than something unique. Thus the discussion could bifurcate into two threads, one about time-series processing and this thread which focuses on data collection mechanisms. I proposed a source of data, which is a periodic self-sampling of the "INFO" metrics. An initial implementation of this would trivially be built by having a periodic timer recursively invoke the INFO command and parse the results. Long term, I envision this as driving a re-architecting of stat collection within the Valkey universe to avoid the serialize/deserialize overhead of this approach, gaining efficiency and therefore usability. This would also provide a degree of uniformity in format and semantics for info stats as well as a reflection mechanism (i.e., command getkeysandflags for info stats) that could drive more generic tools like a grafana connector. @mwarzynski proposed a source of data which is to tap info the command processing and invoke a LUA script with the command, it's arguments and execution time. This is simple, but expensive in that it's going to duplicate a lot of the work that the core already does for you. For example, rather than a single tap-in point for all commands, why not have a per-command tap-in point? I mean the ability to establish a separate LUA script to be invoked for each command.. This would avoid needless LUA execution for commands that aren't of interest. Also the per-command LUA scripts run faster because the command parsing is already completed. With that thought in mind there are other potential tap-in points. For example, leveraging the ACL infrastructure would allow you to tap into core code that validates read and write access for keys independent of the commands., again something that reduces redundant parsing overhead. I'm sure there will be more points that would prove profitable. If we're in a world of multiple data sources and LUA scripts, then we should think about how those different LUA environments interact. Is there a single global LUA environment for all of OBSERVE or is there a need for multiple environments? |
@allenss-amazon If I understand your proposal correctly, you suggest dividing it into two distinct sections.
Implementation of the 'data collection' would be much smaller in scope and would be a great first step towards the support of the full-blown observability pipelines functionality. If that's our plan, then I am fully onboard with this idea. I can revise the proposal to the scope of the “data collection” thread in a manner that allows it to be expanded to support a more comprehensive approach that will then support “data analysis.” @allenss-amazon If you agree, I am going to post a new comment with a new design proposal, and link it at the top of the existing proposal (top of the GH Issue description). I wouldn't replace the current Issue description, because it's worth preserving it for other people as otherwise our discussion in the comments above wouldn't make sense. You also started the discussion about Lua scripts -- pros and cons of specific taps placement. I might need to understand more technical details about Valkey's codebase, and specifically Lua's impact on performance, to get your perspective. So far I thought we might implement efficient filtering in C for the basic fields (e.g. command type) and run Lua only in cases where users will want to have something custom. However, I might have been mistaken, because Lua could be a much more common use case than I assumed, thus the performance penalty would be too large. I probably also lack other important technical details. Thank you for the insight!
I’m also unsure which functionalities I’d be duplicating. Do we already have an easy way to hook into any command executions? I couldn’t find a generic way to do so that covers all command executions. Good news is that our design of taps should be orthogonal to the issue of 'taps placement'. |
|
Yes, let's focus initially on data collection. w.r.t. LUA scripts, my issue isn't about the speed of LUA vs C, but rather about the duplication of work in general. In other words, the core already does substantial processing on commands and keys generically, we can harness that work by adding tap points into the generic command processing infrastructure so that our observability infrastructure need not repeat that work. One thing to think about is that the existing LUA infrastructure in the core assumes it's only being executed -- between -- commands, i.e., as a command itself. Thus some of the LUA functionality may or may not be available to the tap-point infrastructure. For example, I'm skeptical that your LUA function would be able to "call" a Valkey command in order to insert a result into a key (this would invoke the tap recursively?). Also, the issues with replication are interesting to think about -- is this functionality only useful on a primary? Maybe the idea that the results are delivered back into the Valkey keyspace is itself bogus. So as far as data collection is concerned, I think there's substantial thinking required on how to deliver the output of the Tap function back into the Valkey world. |
OverviewThis revised proposal focuses on the "data collection" aspect of the observability pipeline for Valkey. The aim is to establish a foundational system for collecting and processing observability data efficiently, laying the groundwork for future expansion into more advanced data analysis, such as time-series metrics and aggregation. Basic data collection capabilities are powerful enough to expose the useful functionality of polling data and implementing connectors with open-source observability tooling (e.g., Grafana). BackgroundFollowing discussions with Irfan Ahmad at the '24 Valkey Summit and subsequent conversations with @allenss-amazon, we agreed to divide the observability proposal into two distinct phases to better manage its scope and focus on foundational development first:
This document presents the scope of the "Data Collection" phase, which is the groundwork for the observability pipelines feature. Current Observability Limitations in ValkeyValkey's existing observability tools, including
These commands lack the flexibility for in-depth, customizable observability exposed directly within the valkey-server instance, such as filtering specific events, sampling data, executing custom processing steps, aggregating metrics over time windows. Proposal: Data CollectionThe "Data Collection" phase introduces a lightweight mechanism for observing Valkey's internal events through taps. These taps enable capturing and processing data with Lua scripts and retrieving results without overburdening server resources. Core Components1. TapsTaps serve as entry points for collecting specific data units within Valkey, where "data units" is a generic term encompassing various forms of observability data. For example, data units might represent details about command execution or periodically gathered general server state information. Each Tap implementation has a unique name and provides data units of a specific type, reflecting the varied nature of data collected by different Taps. 2. Tap AttachmentUsers can start the data collection from a particular Tap by creating a 'Tap Attachment'. Tap Attachment defines that the data collection functionality should start gathering data from a specific Tap. Optionally, data gathering can be extended with custom processing logic for Tap Attachments using Lua scripts. This enables:
3. Output MechanismThe solution for retrieving processed data via Tap Attachment is yet to be defined but must meet these characteristics:
The output mechanism is yet "to be defined," as the solution must satisfy all of the points outlined above. Commands
|
mwarzynski
changed the title
OBSERVE command for enhanced observability in ValkeyOBSERVE command for enhanced observability in Valkey: Data Collection
|
Hello Folks, For metrics to be usable it is of fundamental importance to have a standard interface to access those metrics. What is the plan ? I read the all GitHub issue but the above is not clear to me. I believe community users today rely on redis_exporter but this is kind of a workaround, and it would be necessary to have an officially supported solution. Thanks |
|
I think this is a good start, the kinds of things that I'd like to see addressed in the next level of refinement is to:
In view of @zioproto's comments, I suspect that retrieving the collected data through the mainthread, i.e., by executing a command, is not the right approach for two reasons. (1) Heisenbergian phenomenon. (2) Valkey native data formats are foreign to standard observability tools, meaning that additional conversion logic has to be provided, which just makes it harder to adopt. I believe that directly supporting more standard data formats would make this more useful. When thinking about the implementation. If we adopt @zioproto's comments and target something like supporting a direct Prometheus compatible HTTP(s) endpoint for data collection, I think we can use this to drive a better implementation, i.e., more light weight. Specifically, processing a non-Valkey format I/O interface is best done with on a separate port using more standard HTTP-ish interface -- which is best implemented with a background thread. What if we made the background thread entirely responsible for processing of data. In other words, the mainthread implementation of a tap is solely to capture some relevant information and put it into a queue for background consumption (some kind of lock-less message queue, like a ring buffer). All of the actual processing of the data, (e.g., LUA script consumption) would be done in the background thread. Something like this would be lightweight enough that it could easily be left on in most production environments with negligible impact on normal mainthread processing. What do you think about this? |
|
@allenss-amazon Thank you for your detailed feedback and thoughtful points! Below are responses to each aspect of your comments. Specifics of Taps and Tap Attachments
You’re correct that the design allows multiple attachments to a single tap. Let’s clarify and refine the specifics:
This approach focuses on simplicity with a room for future extensibility. What are your thoughts? Lua Script Argument Specification
Each tap emits a different type of data unit, which can then be processed by a Lua script that conforms to a defined function signature. Here’s how we could handle this:
To validate this approach, I plan to implement a proof of concept. Since you have more experience with running Lua scripts in Valkey, I’d appreciate your input or suggestions for improvement. I’m also open to adjusting the proposal if needed. It won't be easy and I might have a better proposal after the prototype implementation. There will be many challenges I didn't think about yet, like e.g. validation of the provided Lua code by users. Tap Attachment Behavior Without Lua Scripts
Attachments without Lua scripts are valid and will function as passthrough mechanisms. The data collected from the Tap will flow directly to the output mechanism without custom processing. This could be useful for users who want raw data or don’t need any filtering, custom processing or sampling. In other words, the default behavior for no Lua script will be to emit raw Tap data as-is. Data Retrieval via Commands vs. Standard Observability Formats
You’ve raised excellent points regarding performance, the Heisenbergian phenomenon, usability, and integration with standard observability tools. Here’s my thinking:
Prometheus and Aggregation Prometheus primarily works with aggregated data, while our current design outputs raw, non-aggregated data. To bridge this gap in the initial phase, users could deploy a separate collector service to:
Once the second phase (data analysis) is complete, we can revisit this and consider native OpenMetrics HTTP integration. (cc: @zioproto) @allenss-amazon What do you think? Is AWS going to allow us to use the OBSERVE functionality through the commands (e.g. is there some kind of a command whitelist)? Would AWS allow the Clients to have access to another external port in order to fetch the data? Apologies in advance, because in this section I was focused more on the topic of 'is this feature going to be usable on major clouds' rather then the excellent angles you provided in your comment. Background Thread for Data Processing
This is a great suggestion, and I agree we should aim for negligible performance impact. I need more time to explore this aspect in detail, but in general I like your proposal -- yes, we could use a queue to offload data capture from the main thread to a background thread. The background thread would handle Lua script execution and data output. I need to further explore the practicality and trade-offs and implementation feasibility of these approaches. Your input or pointers to similar Valkey implementations would be invaluable for guiding this process. @allenss-amazon Thank you again for your insights! Looking forward to your thoughts on these approaches. I am open to any suggestions you might have. |
|
I'm skeptical that having a separate LUA instance for each tap attachment is the best approach. I can imagine situations where the desired metrics are synthesized by correlating events from different taps and then aggregated. With separate LUA instances you can't do that. Seems to me that once you let something like LUA into the picture it really wants to absorb as much of your logic as possible, so that flexibility is maintained. In other words, the hard coded functionality in Valkey should be associated with how to deliver to LUA the right information (taps, tap attachments) but that LUA is in charge of consuming that data and maintaining whatever correlations, aggregations, etc. are desired. Then all that's required is some way to retrieve information from LUA. In this model, a "tap attachment" is really a LUA workspace. When you attach the workspace to a tap, events from that tap are delivered to the workspace for processing. It's the workspace's job to do that processing (in the background), maintaining any global data structures it sees fit to maintain. Then all that's required is some way to retrieve results. No reason we can't have multiple retrieval mechanisms, i.e., a command to start with and maybe some background HTTP-ish thing later. A retrieval is simply an invokation of a LUA routine that delivers a set of results which are then output. With this framework, what's required in the spec is to get specific with the details of delivering events to a lua workspace. Some description of how to limit the resources consumable by a LUA workspace (how to prevent consuming too much memory, how to get out of infinite loops, etc.), and the retrieval mechanism -- initially the retrieve subcommand you have. |
|
I am a bit late for this long discussion and tried to catchup with the long thread. Overall I do agree we should invest in better observably tooling support but I am not sure the data ingestion should really be a core component. |
|
Also @mwarzynski do we plan to continue the discussion over the proposed RFC? I think it is better to do so. |
|
@mwarzynski this is great initiative! I had similar discussions with @ranshid but you beat me to it 😄 A general comment: I think that any data collection implemented within Valkey, should be based on Open Telemetry (especially using the "Collector" design approach which decouples the backend from the data collection) Open Telemetry guidelines for Redis / Valkey can be found: (s/redis/valkey/g) A note about the implementation: Ideally, this should be outsourced to a module (preferably written in Rust - which supports multiple crates for open-telemetry, so histograms, spans, counters etc are already supported which leaves us to focus on the data collection points only) Obviously, Valkey's module API should be extended to support your complete vision, but we already have a good starting point (For example, using IMO, the advantages of implementing this as a core module outweigh the disadvantages. For example:
|
asafpamzn
added
the
client-changes-needed
Original proposal was divided into two sections: data collection and data analysis. This issue is re-scoped to reflect the design of 'data collection'. New proposal is here: #1167 (comment).
I am leaving the original proposal for full blown observability pipelines unchanged. If you are not interested in previous versions and all of the historical context, please go directly to the new proposal here: #1167 (comment).
TLDR, I propose to improve observability for Valkey, like built-in RED time-series metrics
Overview
This proposal outlines a new
OBSERVEcommand to improve Valkey’s observability capabilities. By enabling advanced time-series metrics, custom gathering pipelines, and in-server data aggregation,OBSERVEwill equip Valkey users with first-class monitoring commands for granular insight into server behavior and performance.Background
After discussions with Irfan Ahmad, an attendee at the '24 Valkey Summit, I developed this initial proposal to introduce native observability pipelines within Valkey. Currently, Valkey lacks comprehensive, customizable observability tools embedded directly within the server, and this proposal aims to fill that gap.
Note: This proposal is a work in progress. Feedback on the overall approach and any preliminary design concerns would be greatly appreciated.
Current Observability Limitations in Valkey
Currently, Valkey’s observability relies on commands like
MONITOR,SLOWLOG, andINFO.While useful, these commands have limitations:
MONITOR: Streams every command, generating high data volume that may overload production environments.SLOWLOG: Logs only commands exceeding a set execution time, omitting quick operations and general command patterns.INFO: Provides server statistics but lacks detailed command- and key-specific insights.These commands lack the flexibility for in-depth, customizable observability exposed directly within the valkey-server instance,
such as filtering specific events, sampling data, executing custom processing steps, aggregating metrics over time windows.
Feature proposal
Problem statement and goals
The proposed
OBSERVEcommand suite will bring observability as a core Valkey feature. Through user-defined “observability pipelines,” Valkey instances can produce detailed insights in a structured, efficient manner. These pipelines will be customizable to support diverse use cases, providing users with foundational building blocks for monitoring without overwhelming server resources. This new functionality could be enhanced with integration with tools like Prometheus and Grafana for visualization or alerting, though its fully customizable and primary purpose is in-server analysis.Proposed solution -- Commands
The
OBSERVEcommand set introduces the concept of observability pipelines — user-defined workflows for collecting, filtering, aggregating, and storing metrics.Core Commands
OBSERVE CREATE <pipeline_name> <configuration>Creates an observability pipeline with a specified configuration. Configuration details, specified in the next section, define steps such as filtering, partitioning, sampling, and aggregation.
Pipeline and it's configuration is persisted in the runtime memory (i.e. user needs to re-create the pipeline after server restart).
OBSERVE START <pipeline_name>Starts data collection for the specified pipeline.
OBSERVE STOP <pipeline_name>Stops data collection for the specified pipeline.
OBSERVE DELETE <pipeline_name>Deletes the pipeline and its configuration.
OBSERVE RETRIEVE <pipeline_name>Retrieves collected data. Alternatively, GET could potentially serve for this function, but further design discussion is needed.
OBSERVE LOADSTEPF <step_name> <lua_code>Allows defining custom processing steps using Lua, for cases where built-in steps do not meet needed requirements.
Pipeline configuration
Pipelines are configured as chains of data processing stages, including filtering, aggregation, and output buffering. Format is similar to the Unix piping.
Key stages in this pipeline model include:
filter(f): Filters events based on defined conditions (e.g., command type).partition(f): Partitions events according to a function (e.g., by key prefix).sample(f): Samples events at a specified rate.map(f): Transforms each event with a specified function.window(f): Aggregates data within defined time windows.reduce(f): Reduces data over a window via an aggregation function.output(f): Directs output to specified sinks.Example configuration syntax:
Output
The goal is to capture time-series metrics within the defined pipeline outputs, f.e. for the pipeline above it would be structured as follows:
It remains uncertain whether storing output data in a format compatible with direct retrieval via GET (or another existing command) will be feasible. Consequently, we might need to introduce an
OBSERVE RETRIEVE <since_offset>command for clients polling results data. This command would provide:Here, offset represents the sequence number of items produced by the pipeline, including any items removed due to buffer constraints. This approach allows clients to poll for results while adjusting their polling frequency based on the lag_detected flag. If lag_detected is true, clients would be advised to increase polling frequency to reduce data loss.
Use-Case Examples
Below are examples of how the proposed
OBSERVEcommand and pipeline configurations could be used to address variousobservability needs.
Counting Specific Commands Per Minute with Buffer Size
Use Case: Count the number of
GETcommands executed per minute.Pipeline Creation:
Explanation: This pipeline filters for
GETcommands, counts them per every minute, and stores the countsin a time-series key
get_command_countwith a buffer size of 1440 (e.g., one day's worth of minute-level data).Hot Key Analysis
Use Case: Identify and monitor the most frequently accessed keys within a certain time window, allowing for proactive load management and identification of potential bottlenecks.
Pipeline Creation:
Explanation: This pipeline filters for sampled 0.5% of GET commands, partitions events by the accessed key, and aggregates their counts in one-minute intervals.
The map_top_keys(10) step then selects the top 10 most frequently accessed keys in each interval along with the access counts.
The result is stored as a time-series in hot_keys with a buffer size of 60, retaining one hour of hot key data.
Average Latency Per Time Window with Buffer
Use Case: Monitor average latency of
SETcommands per minute.Pipeline Creation:
Explanation: This pipeline filters for
SETcommands, extracts their latency, aggregates the average latency everyminute, and stores it with a buffer size of 720 (e.g., 12 hours of minute-level data).
Client Statistics
Use Case: Gather command counts per client for
GETandSETcommands, sampled at 5%.Pipeline Creation:
Explanation: This pipeline filters for
GETandSETcommands, samples 5% of them, extracts client information, counts commands per client every minute, and stores the data under
client_statswith a buffer size of 1440.Error Tracking
Use Case: Monitor the number of errors occurring per minute.
Pipeline Creation:
Explanation: This pipeline filters events of type 'error', counts them every minute, and stores the totals in
tota l_errorswith a buffer size of 1440.TTL Analysis
Use Case: Analyze the average TTL of keys set with
SETEXcommand per minute.Pipeline Creation:
Explanation: This pipeline filters for
SETEXcommands, extracts the TTL values, calculates the average TTL everyminute, and stores it in
average_ttlwith a buffer size of 1440.Distribution of Key and Value Sizes
Use Case: Create a histogram of value sizes for
SETcommands.Pipeline Creation:
Explanation: This pipeline filters for
SETcommands, extracts the size of the values, aggregates them into histogram buckets every minute, and stores the distributions with a buffer size of 1440.
Feedback Request
Feedback is requested on the following points:
OBSERVEcommand align with your vision for Valkey’s observability?Let's first reach the consensus for the 'Feature Scope'. If the answer is yes, we can discuss the designs.
I am ready to commit to building this feature as soon as the designs are accepted, even in draft form.
Thank you for your time and consideration. I look forward to discussing this proposal further.
The text was updated successfully, but these errors were encountered: