Hybrid Search Sorting Issue with Size Specified

[jdomkline]

What is the bug?

With a from of 0 and an arbitrary size set, the Hybrid Search will give incorrect sorting results. The problem can appear and disappear based on changes to the size limit, such as setting size at 8 instead of 10.

It can be seen that the sort is actually correct however during document retrieval the wrong document is grabbed, not the document indicated by the sort (please see detailed screen caps below).

How can one reproduce the bug?

Run a DSL Hybrid Query with various sizes sorting on a field and the _id (please see detailed screen caps below).

What is the expected behavior?

During document retrieval, the correct document is returned as specified by the sort.

What is your host/environment?

AWS 2.17
{
"name": "0bf1a5d4ef04a73480c61776916c50c3",
"cluster_name": "302642896912:productsearch",
"cluster_uuid": "vghL7JqlRQijNk4yvpJk_Q",
"version": {
"number": "7.10.2",
"build_type": "tar",
"build_hash": "unknown",
"build_date": "2024-11-18T04:22:32.407132088Z",
"build_snapshot": false,
"lucene_version": "9.11.1",
"minimum_wire_compatibility_version": "7.10.0",
"minimum_index_compatibility_version": "7.0.0"
},
"tagline": "The OpenSearch Project: https://opensearch.org/"
}

Do you have any screenshots?

The sort retrieval problem example.

Example of correct results with minor adjustment to the size, from 8 to 10.

Website example

Do you have any additional context?

The case can be repeated over and over... setting the size back and forth.

[minalsha]

@martin-gaievski could you please look into this bug?

[jdomkline]

I saw progress is going on with the paging option (from > 0). It could be this was found already. With paging size would play a key part.

[martin-gaievski]

@jdomkline can you please share additional information about the issue:

• configuration of your cluster (howe many data nodes)
• index mapping (number of primary and replica shards)
• search query - what are exact sub-queries under hybrid clause, what is the aggregation(s). best case if you can share the whole search query

[jdomkline]

@martin-gaievski this is a QA cluster so it is fairly small.

1-AZ with 2 data nodes.

"number_of_shards": "5",
...
"number_of_replicas": "1"

query.json

Luckily the sort info provided in the result allows a post processing to gather the missing doc and correct the result. For my use case this is an ok workaround. For a much larger size I can see it not working.

[martin-gaievski]

reproduced on our end, thank you for detailed steps. Team is checking what is possible root cause and if workaround is possible

[martin-gaievski]

@jdomkline this issue is a known problem in the implementation of the sort function. We have fixed it recently, and the code change will be part of the upcoming 2.19 version. You can find the corresponding code change in this PR: #1043.

[jdomkline]

Hi @martin-gaievski thanks for the link. I see this case when I'm not including a sort at all. If I set the size at 10 a certain document comes back in position 8. When I use size at 20, this same document comes back in position 11. From what I read, it seems that PR: #1043. has addressed this also. Please share your thoughts.

This would need to be fixed if from > 0 is ever allowed (vs searchAfter) for Hybrid queries.

[martin-gaievski]

@jdomkline fix in #1043 addresses issue of field values being mismatched between source and sort sections in the search hits collection. In other words, if you use sort in your queries you need to have that change.

Scenario where a document's position changes depending on the size value can be an intended behavior of hybrid query. To form the final search results list, we take up to size number of documents (which is the minimum between matched_documents and size) from each shard and then combine them at the coordinator level. The larger the size, the more documents will be selected from each shard. In your scenario, when the size was increased from 10 to 20, it's possible that the combined contribution of document X became less significant for different sub-queries, based on the query and data distribution

[jdomkline]

@martin-gaievski I'm unclear as this was the exact same query with both sizes (10 and 20), how paging with Hybrid will ever work without providing duplicates. Once people can do from > 0 they will also be setting sizes. Is searchAfter the only way paging can work without duplicates then? (assuming no need for PIT)

[martin-gaievski]

for pagination in hybrid queries, users will need to set the global window size with the new parameter pagination depth. The starting position will be controlled by the from parameter (which acts like a cursor), and size will control the number of records extracted by the query. To navigate to the next page, users will need to change the from value. The actual number of records selected from each shard is controlled by the pagination depth. As long as that value remains constant, changes to size will not create duplicates.

[jdomkline]

@martin-gaievski that is awesome! Got it thanks

[martin-gaievski]

Closing this issue, fix is provided in #1043