Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support conditional script update of documents in the OpenSearch sink #3563

Open
graytaylor0 opened this issue Oct 30, 2023 · 0 comments
Open

Comments

@graytaylor0
Copy link
Member

Is your feature request related to a problem? Please describe.
As a user of the OpenSearch sink, I would like to conditionally update documents by comparing the existing document in OpenSearch with the updated document.

Describe the solution you'd like
An option to specify a script parameter that will apply to update and upsert operation in the sink (https://opensearch.org/docs/latest/api-reference/document-apis/update-document/#script-example).

This requires that the incoming update documents contain the fields that are referenced in the script.

Describe alternatives you've considered (Optional)
One alternative to having OpenSearch handle this would be to have Data Prepper handle it, which would mean an extra read of each document with an update or upsert operation, and then relying on Data Prepper expression as the condition instead of the script. the benefit of this approach would be that we could support comparisons on Data Prepper metadata, instead of requiring the data be in the Event.

Additional context
Add any other context or screenshots about the feature request here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

No branches or pull requests

1 participant