Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Elastic Search _shard_doc equivalent #17064

Open
DwaneHall opened this issue Jan 21, 2025 · 0 comments
Open

Add support for Elastic Search _shard_doc equivalent #17064

DwaneHall opened this issue Jan 21, 2025 · 0 comments
Labels
enhancement Enhancement or improvement to existing feature or request untriaged

Comments

@DwaneHall
Copy link

DwaneHall commented Jan 21, 2025

Is your feature request related to a problem? Please describe

When paginating search results using the search_after parameter there's no guarantee that results are returned from the same shard in the same order. We notice regular duplicates across pages when results are serviced from a different shard (2 replicas). It's cumbersome to have to make a request to the PIT API to first retrieve a PIT ID to then use it in subsequent queries. It would be nice to be able to specify a default tiebreaker (i.e use primaries for searching) if you know what shards/replicas you'd like to use . Elastic Search appears to solve this issue with the _shard_doc parameter and being able to specify it directly as a sort parameter but it's not supported in OS (https://forum.opensearch.org/t/point-in-time-errors/14068).

https://discuss.elastic.co/t/elastic-no-mapping-found-for-shard-doc-in-order-to-sort-on/268188/2
https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html

Describe the solution you'd like

Support for _shard_doc (or equivalent) sorting instead of having to call the PID api first if you know upfront what shards you'd prefer to use for pagination. Elastic also presents a "preference" search parameter https://www.elastic.co/guide/en/elasticsearch/reference/1.7/search-request-preference.html

Related component

No response

Describe alternatives you've considered

PIT api

Additional context

No response

@DwaneHall DwaneHall added enhancement Enhancement or improvement to existing feature or request untriaged labels Jan 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or improvement to existing feature or request untriaged
Projects
None yet
Development

No branches or pull requests

1 participant