Skip to content

Commit

Permalink
Add pipeline metrics to Node Stats API (#16839)
Browse files Browse the repository at this point in the history
This commit introduces three new metrics per pipeline in the Node Stats API:
- workers
- batch_size
- batch_delay

```
{
  ...
  pipelines: {
    main: {
      events: {...}, 
      flow: {...}, 
      plugins: {...}, 
      reloads: {...}, 
      queue: {...}, 
      pipeline: {
        workers: 12,
        batch_size: 125,
        batch_delay: 5,
      }, 
    }
  }
  ...
}
```
  • Loading branch information
kaisecheng authored Jan 3, 2025
1 parent 531f795 commit de6a6c5
Show file tree
Hide file tree
Showing 3 changed files with 22 additions and 1 deletion.
15 changes: 15 additions & 0 deletions docs/static/monitoring/monitoring-apis.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -726,6 +726,11 @@ Example response:
},
"queue" : {
"type" : "memory"
},
"pipeline": {
"workers": 4,
"batch_size": 125,
"batch_delay": 50
}
},
"test2" : {
Expand Down Expand Up @@ -807,6 +812,11 @@ Example response:
},
"queue" : {
"type" : "memory"
},
"pipeline": {
"workers": 4,
"batch_size": 125,
"batch_delay": 50
}
}
}
Expand Down Expand Up @@ -969,6 +979,11 @@ Example response:
"events_count": 0,
"queue_size_in_bytes": 3885,
"max_queue_size_in_bytes": 1073741824
},
"pipeline": {
"workers": 4,
"batch_size": 125,
"batch_delay": 50
}
}
}
Expand Down
7 changes: 6 additions & 1 deletion logstash-core/lib/logstash/api/commands/stats.rb
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,12 @@ def report(stats, extended_stats = nil, opts = {})
:outputs => plugin_stats(stats, :outputs)
},
:reloads => stats[:reloads],
:queue => stats[:queue]
:queue => stats[:queue],
:pipeline => {
:workers => stats[:config][:workers],
:batch_size => stats[:config][:batch_size],
:batch_delay => stats[:config][:batch_delay],
}
}
ret[:dead_letter_queue] = stats[:dlq] if stats.include?(:dlq)

Expand Down
1 change: 1 addition & 0 deletions logstash-core/spec/logstash/api/commands/stats_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,7 @@
:plugins,
:reloads,
:queue,
:pipeline
)
end
it "returns event information" do
Expand Down

0 comments on commit de6a6c5

Please sign in to comment.