Redis for logs and statistics

Logs to Redis

Docs moved to: https://openvstorage.gitbooks.io/framework/content/docs/log.html

Statistics in redis

Each component drops his stats into his own stats queue where the consumers can pick them up for insertion into for example influxdb.

Available stat queues

ovs_stats_volume
ovs_stats_alba
...

The publish interval should be configurable via a reloadable parameter. The stats itself are raw numbers, no mean, median, variance, etc

$measurement,tag1=value1,tag2=value2 measurement1=value1,measurement=value2 epoch_timestamp_in_microseconds

Example:

readcache,volume=<volume_id> misses=0,hits=0 1453736430.572463
writebuffer,volume=<volume_id> misses=0,hits=70 1453736430.572463
data,volume=<volume_id>	read=2064384,written=0,transferred=2064384,backend_read=0,backend_written=0 1453736430.572463
metadata,volume=<volume_id> misses=1,hits=503 1453736430.572463
operations,volume=<volume_id> read=54,write=0,backend_read=0,backend_write=0 1453736430.572463

QUESTIONS:

• We need to think all used fields and tags through, as they have a huge impact on what an InfluxDB instance can handle. At this moment (0.6.9/10), InfluxDB is quite memory hungry and if the above items are not used carefully (or worse, changed), things can you out of control.
• Use protobuf or another serialization format instead of plain text? This can for example prevent certain types of bugs by keeping the definitions of records in one place.