Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update options and add more examples for add_entries processor #7412

Merged
merged 5 commits into from
Jul 2, 2024
Merged

Update options and add more examples for add_entries processor #7412

merged 5 commits into from
Jul 2, 2024

Conversation

oeyh
Copy link
Contributor

@oeyh oeyh commented Jun 20, 2024

Description

  • Adds append_if_key_exists option
  • Adds update to key option
  • Re-orders a few existing options
  • Adds examples for common use cases

Issues Resolved

Closes #7411

Version

Data Prepper latest

Checklist

  • By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
    For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@oeyh oeyh changed the title Update options and add more examples Update options and add more examples for add_entries processor Jun 20, 2024
@hdhalter hdhalter added data-prepper 2.8 4 - Doc review PR: Doc review in progress labels Jun 24, 2024
@Naarcha-AWS Naarcha-AWS self-assigned this Jun 25, 2024
Naarcha-AWS
Naarcha-AWS requested changes Jun 25, 2024
View reviewed changes
_data-prepper/pipelines/configuration/processors/add-entries.md Outdated Show resolved Hide resolved
_data-prepper/pipelines/configuration/processors/add-entries.md Outdated Show resolved Hide resolved
_data-prepper/pipelines/configuration/processors/add-entries.md Outdated Show resolved Hide resolved
_data-prepper/pipelines/configuration/processors/add-entries.md Outdated Show resolved Hide resolved
_data-prepper/pipelines/configuration/processors/add-entries.md Outdated Show resolved Hide resolved
_data-prepper/pipelines/configuration/processors/add-entries.md Outdated Show resolved Hide resolved
_data-prepper/pipelines/configuration/processors/add-entries.md Outdated Show resolved Hide resolved
_data-prepper/pipelines/configuration/processors/add-entries.md Outdated Show resolved Hide resolved
_data-prepper/pipelines/configuration/processors/add-entries.md Outdated Show resolved Hide resolved
@Naarcha-AWS Naarcha-AWS added 5 - Editorial review PR: Editorial review in progress and removed 4 - Doc review PR: Doc review in progress labels Jun 27, 2024
natebower
natebower requested changes Jul 1, 2024
View reviewed changes
Copy link
Collaborator

@natebower natebower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Naarcha-AWS @oeyh Please see my comments and changes and tag me on the revision to line 200 for approval. Thanks!

_data-prepper/pipelines/configuration/processors/add-entries.md Outdated Show resolved Hide resolved
_data-prepper/pipelines/configuration/processors/add-entries.md Outdated Show resolved Hide resolved
_data-prepper/pipelines/configuration/processors/add-entries.md Outdated Show resolved Hide resolved
_data-prepper/pipelines/configuration/processors/add-entries.md Outdated Show resolved Hide resolved
_data-prepper/pipelines/configuration/processors/add-entries.md Outdated Show resolved Hide resolved
_data-prepper/pipelines/configuration/processors/add-entries.md Outdated Show resolved Hide resolved
_data-prepper/pipelines/configuration/processors/add-entries.md Outdated Show resolved Hide resolved
_data-prepper/pipelines/configuration/processors/add-entries.md Outdated

If `overwrite_if_key_exists` is not set to `true`, the input event will not be changed after going through the processor.

### Example: Append to existing entries
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add the noun? Append what to existing entries?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@natebower We can say "Append values to existing entries".

_data-prepper/pipelines/configuration/processors/add-entries.md Outdated Show resolved Hide resolved
_data-prepper/pipelines/configuration/processors/add-entries.md Outdated

### Example: Append to existing entries

The following example shows how you can configure the processor to overwrite existing entries:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this may be a copy and paste from the previous section. Should it read "The following example shows you how to configure the processor to append [noun] to existing entries"?

Copy link
Contributor Author

@oeyh oeyh Jul 1, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. We can say "...configure the processor to append values to existing entries".

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Edited my suggestion.

natebower
natebower approved these changes Jul 1, 2024
View reviewed changes
Copy link
Collaborator

@natebower natebower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@oeyh @Naarcha-AWS LGTM with noted changes. Thanks!

_data-prepper/pipelines/configuration/processors/add-entries.md Outdated Show resolved Hide resolved
@Naarcha-AWS @natebower
Apply suggestions from code review
1c604f0 
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>
@Naarcha-AWS Naarcha-AWS merged commit 509d8f0 into opensearch-project:main Jul 2, 2024
5 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jul 2, 2024
@github-actions @Naarcha-AWS @natebower
Update options and add more examples for add_entries processor (#7412)
861671e 
* Update options and add more examples

Signed-off-by: Hai Yan <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* A few small edits

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>

---------

Signed-off-by: Hai Yan <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>
Co-authored-by: Naarcha-AWS <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
(cherry picked from commit 509d8f0)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot opensearch-trigger-bot bot mentioned this pull request Jul 2, 2024
Merged
github-actions bot pushed a commit that referenced this pull request Jul 2, 2024
@opensearch-trigger-bot
Update options and add more examples for add_entries processor (#7412) (
6f78e08 
#7609)
leanneeliatra pushed a commit to leanneeliatra/opensearch-documentation-website-forl that referenced this pull request Jul 24, 2024
@oeyh @Naarcha-AWS
@natebower @leanneeliatra
Update options and add more examples for add_entries processor (opens…
1f3cebb 
…earch-project#7412)

* Update options and add more examples

Signed-off-by: Hai Yan <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* A few small edits

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>

---------

Signed-off-by: Hai Yan <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>
Co-authored-by: Naarcha-AWS <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: [email protected] <[email protected]>
sandervandegeijn pushed a commit to sandervandegeijn/documentation-website that referenced this pull request Jul 30, 2024
@oeyh @Naarcha-AWS
@natebower @sandervandegeijn
Update options and add more examples for add_entries processor (opens…
6f47d61 
…earch-project#7412)

* Update options and add more examples

Signed-off-by: Hai Yan <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* A few small edits

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>

---------

Signed-off-by: Hai Yan <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>
Co-authored-by: Naarcha-AWS <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Sander van de Geijn <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@natebower natebower natebower approved these changes

@Naarcha-AWS Naarcha-AWS Naarcha-AWS approved these changes

@hdhalter hdhalter Awaiting requested review from hdhalter

@kolchfa-aws kolchfa-aws Awaiting requested review from kolchfa-aws kolchfa-aws is a code owner

@vagimeli vagimeli Awaiting requested review from vagimeli

@AMoo-Miki AMoo-Miki Awaiting requested review from AMoo-Miki AMoo-Miki is a code owner

@dlvenable dlvenable Awaiting requested review from dlvenable dlvenable is a code owner

@stephen-crawford stephen-crawford Awaiting requested review from stephen-crawford

@epugh epugh Awaiting requested review from epugh epugh is a code owner

Assignees

@Naarcha-AWS Naarcha-AWS

Labels
5 - Editorial review PR: Editorial review in progress backport 2.15 data-prepper 2.8
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[DOC] Update add_entries processor docs for Data Prepper
4 participants
@oeyh @Naarcha-AWS @natebower @hdhalter