updated docs #588
updated docs #588
Conversation
|
Minimum allowed coverage is Generated by 🐒 cobertura-action against 5d48bed |
| Elastic Serverless Forwarder ensures "at-least-once" delivery of the forwarded message. | ||
|
|
||
| The `Continuing quque` and `Replay queue` are programmatically populated by Elastic Serverless Forwarder. | ||
| This happens for the `Continuing quque` in the last two minutes of the lambda execution, if any message in the original trigger is left to be processed before that time (including the original triggers and the continuing and replay queues themselves). |
There was a problem hiding this comment.
| This happens for the `Continuing quque` in the last two minutes of the lambda execution, if any message in the original trigger is left to be processed before that time (including the original triggers and the continuing and replay queues themselves). | |
| This happens for the `Continuing queue` in the last two minutes of the lambda execution, if any message in the original trigger is left to be processed before that time (including the original triggers and the continuing and replay queues themselves). |
Hmm does This mean the creation of the queues?
There was a problem hiding this comment.
no, This refers to "programmatically populated"
Co-authored-by: kaiyan-sheng <[email protected]>
Co-authored-by: kaiyan-sheng <[email protected]>
| It is not possible to define directly the ID of the {aws} VPC the Elastic Serverless Forwarder lambda belongs to. | ||
| The {aws} VPC is defined instead by its security group IDs and subnet IDs. |
There was a problem hiding this comment.
It might be worth explaining why it is not possible and add a link to the AWS docs.
As additional note, I do not think this information is enough to close out #488 as there are more pieces involved in making ESF work with PrivateLink, apart from the VPC settings (e.g VPC endpoints)
There was a problem hiding this comment.
VPC endpoints are documented here
Let's move the content to this section! (and add the requirement for an EC2 VPC endpoint if cloudwatch logs are used as input)
I will add a link to https://www.elastic.co/guide/en/cloud/current/ec-traffic-filtering-vpc.html as well
do you have any hint about what @Udayel mentioned in #488?
document few important parameters like
BOTO_DISABLE_COMMONNAME=False
I remember it was something related to a version bump of the ESF boto dependency, but I never hit the issue and I'm not sure if it still applies
| [[at-least-once-delivery]] | ||
| == At-least-once delivery | ||
|
|
||
| Elastic Serverless Forwarder ensures "at-least-once" delivery of the forwarded message. |
There was a problem hiding this comment.
I think this statement should be in the overview, where the Replay queue and Continuing queue are introduced for the first time.
| @@ -79,6 +79,26 @@ The forwarder can ingest logs contained within the payload of an Amazon SQS body | |||
|
|
|||
| You can set up a separate SQS queue for each type of log. The config parameter for {es} output `es_datastream_name` is mandatory. If this value is set to an {es} data stream, the type of log must be correctly defined with configuration parameters. A single configuration file can have many input sections, pointing to different SQS queues that match specific log types. | |||
|
|
|||
| [discrete] | |||
| [[at-least-once-delivery]] | |||
| == At-least-once delivery | |||
There was a problem hiding this comment.
I think this section might be easier to digest if we split it in two, providing details for the two queues in two separate sections/sub-sections (e.g "More details about the Replay queue" or "Replay queue internals).
Each section can follow a pattern like the following:
- What part this queue plays in ensuring at-least-once delivery
- When ESF sends messages to this queue
- Failures/DLQ scenarios
I would also add a note about the recent changes we made, removing sqs.deleteMessage calls and their impact.
* Edit ESF docs added by PR #588 * Resolve open questions * lint fix --------- Co-authored-by: Andrea Spacca <[email protected]>
Docs
What does this PR do?
Fixes #488
Fixes #521
Fixes #522
Why is it important?
Better documentation improves UX
Checklist
- [ ] My code follows the style guidelines of this project- [ ] I have commented my code, particularly in hard-to-understand areas- [ ] I have made corresponding change to the default configuration files- [ ] I have added tests that prove my fix is effective or that my feature works- [ ] I have added an entry inCHANGELOG.mdAuthor's Checklist
How to test this PR locally
Related issues
Use cases
Screenshots
Logs