Doc: Add tutorial for filter-elastic_integration #15932
Conversation
Co-authored by: Sam Wickline <[email protected]>
karenzone
force-pushed
the
15871-tutorial
branch
from
b08d864 to
55d5045
Compare
|
run docs-build |
There was a problem hiding this comment.
I know this is in draft form, but I'm hoping some suggestions can steer this in a way that de-elevates the implementation details about Ingest Node Pipelines and elevates the plugin's true purpose: running Elastic Integrations inside Logstash.
I'm also hoping to trim out some security-bypassing configuration from the config examples.
Yes, still very much in draft form so that we can get a stake in the ground. But not too early to register directional comments, so thanks for this. |
karenzone
force-pushed
the
15871-tutorial
branch
from
7cec523 to
63962dc
Compare
| hosts => "{es-host}:9200" | ||
| password => "changeme" | ||
| user => "elastic" | ||
| cacert => "/usr/share/logstash/config/certs/ca-cert.pem" |
There was a problem hiding this comment.
Modern versions of the Elasticsearch Output plugin still support legacy settings like cacert, but they also support (and prefer) the normalized SSL settings that match the ones in the Elastic Integration Filter.
| cacert => "/usr/share/logstash/config/certs/ca-cert.pem" | |
| ssl_certificate_authorities => "/usr/share/logstash/config/certs/ca-cert.pem" |
| This tutorial walks you through adding the {integrations-docs}/crowdstrike-intro[Crowdstrike integration], using {ls} to | ||
| remove the `_version` field, and then sending the data to {ess} or self-managed {es}. |
There was a problem hiding this comment.
I believe that removing the _version field was a workaround for the specific ingest pipeline setting the _version field on ingest document instead of on the ingest document's metadata. When it gets put on the metadata, our filter correctly propagates it to the right places on the resulting event so that downstream ES output can choose to use it or not depending on its configuration.
My worry here is that people will copy/paste this config and assume that it is necessary.
There was a problem hiding this comment.
ToDo: Come up with a better example that won't mislead users
|
Hey @karenzone, do you think this tutorial is close to being published? We're starting to promote the integrations filter to customer and internally via enablement sessions. The current documentation provides no guidance as to how to actually get the pipelines from our integrations into Logstash, and this tutorial seems to bridge that gap. If it's close to being published I can share that'll have the relevant guidance coming soon. |
Co-authored-by: Ry Biesemeyer <[email protected]>
|
@jamiehynds, this work is being picked up again, and should be published soon. |
| and uses them to apply the transformations from Elastic integrations. This allows you to to further process events in the Logstash pipeline before sending them to their | ||
| configured destinations. | ||
|
|
||
| This tutorial walks you through adding the {integrations-docs}/crowdstrike-intro[Crowdstrike integration], using {ls} to |
There was a problem hiding this comment.
@flexitrev, would you like to highlight a different integration for this tutorial and use case?
| [[ea-integrations-tutorial]] | ||
| === Tutorial: Using the {ls} `elastic_integration filter` to extend Elastic {integrations} | ||
| ++++ | ||
| <titleabbrev>Tutorial: {ls} `elastic_integration filter`</titleabbrev> |
There was a problem hiding this comment.
| <titleabbrev>Tutorial: {ls} `elastic_integration filter`</titleabbrev> | |
| <titleabbrev>Tutorial: {ls} `elastic_integration` filter</titleabbrev> |
|
|
||
| You can use {ls} to transform events collected by {agent} and paired with an {integrations-docs}[Elastic integration]. | ||
| You get the benefits of Elastic integrations--such as the simplicity of ingesting data from a wide variety of data | ||
| sources and ensuring compliance with the {ecs-ref}/index.html[Elastic Common Schema (ECS)]--combined with the extra |
There was a problem hiding this comment.
-- isn't rendered properly?
|
|
||
| * <<ea-integrations-fleet>> | ||
| * <<ea-integrations-create-policy>> | ||
| * <<ea-integrations-pipeline>> |
There was a problem hiding this comment.
Don't we need to setup LS instance before the LS policy? - I haven't fully followed the steps, not sure what experience I get when setting the policy first.
|
📃 DOCS PREVIEW ✨ https://logstash_bk_15932.docs-preview.app.elstc.co/diff |
Starts with tutorial content from gdoc and converts it to asciidoctor format.
PREVIEW: https://logstash_bk_15932.docs-preview.app.elstc.co/guide/en/logstash/master/ea-integrations-tutorial.html
Closes: #15871