Skip to content

Commit

Permalink
Document presets for performance tuning to ES output (#749)
Browse files Browse the repository at this point in the history
* Document presets for performance tuning to ES output

* Typo and link fix

* Address review comments

(cherry picked from commit ffd8a6d)
  • Loading branch information
kilfoyle authored and mergify[bot] committed Dec 12, 2023
1 parent 5e75491 commit 4998bf7
Showing 1 changed file with 106 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,16 @@ output is set in the <<agent-policy,agent policy>>.
| When this setting is on, {agent}s use this output to send <<monitor-elastic-agent,agent monitoring data>> if no other output is set in the <<agent-policy,agent policy>>.

Sending monitoring data to a remote {es} cluster is currently not supported.

// =============================================================================

|
[id="es-agent-performance-tuning"]
**Performance tuning**

| Choose one of the menu options to tune your {agent} performance when sending data to an {es} output. You can optimize for throughput, scale, latency, or you can choose a balanced (the default) set of performance specifications. Refer to <<es-output-settings-performance-tuning-settings,Performance tuning settings>> for details about the setting values and their potential impact on performance.

You can also use the <<es-output-settings-yaml-config,Advanced YAML configuration>> field to set custom values. Note that if you adjust any of the performance settings described in the following **Advanced YAML configuration** section, the **Performance tuning** option automatically changes to `Custom` and cannot be changed.
|===

[[es-output-settings-yaml-config]]
Expand Down Expand Up @@ -118,4 +128,100 @@ include::../elastic-agent/configuration/outputs/output-shared-settings.asciidoc[

|===


[[es-output-settings-performance-tuning-settings]]
== Performance tuning settings

.Performance tuning preset values
[cols="1,1,1,1,1"]
|===
|Configuration |Balanced|Optimized for Throughput |Optimized for Scale |Optimized for Latency

|`bulk_max_size`
|1600
|1600
|1600
|50

|`worker`
|1
|4
|1
|1

|`queue.mem.events`
|3200
|12800
|3200
|4100

|`flush.min_events`
|1600
|1600
|1600
|2050

|`flush.timeout`
|10
|5
|20
|1

|`compression`
|1
|1
|1
|1

|`idle_timeout`
|3
|15
|1
|60
|===

For descriptions of each setting, refer to <<es-output-settings-yaml-config,Advanced YAML configuration>>. For the `queue.mem.events`, `flush.min_events` and `flush.timeout` settings, refer to the {filebeat-ref}/configuring-internal-queue.html[internal queue configuration settings] in the {filebeat} documentation.

.Potential performance effect
[cols="1,1,1,1,1"]
|===
|Configuration |Balanced|Optimized for Throughput |Optimized for Scale |Optimized for Latency

|Stateful throughput
|3x
|5x
|3x
|1x

|Serverless throughput
|5-10x
|10-20x
|5-10x
|1x

|Connections
|0.3x
|4x
|0.04x
|1x

|Network traffic
|0.1x
|0.1x
|0.05x
|0.1x

|High-throughput queue latency
|1x
|1x
|1x
|1x

|Low-throughput queue latency
|10x
|5x
|20x
|1x
|===

:type!:

0 comments on commit 4998bf7

Please sign in to comment.