Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Federated search added #567

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Nymuxyzo
Copy link
Contributor

Pull Request

Related issue

Fixes #558

What does this PR do?

  • Federated search added

PR checklist

Please check if your PR fulfills the following requirements:

  • Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
  • Have you read the contributing guidelines?
  • Have you made sure that the title is accurate and descriptive of the changes?

Copy link

codecov bot commented Oct 11, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (085793f) to head (6b51188).

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #567   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           10        10           
  Lines          737       737           
=========================================
  Hits           737       737           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Nymuxyzo Nymuxyzo changed the title Federated search added Federated search added (#558) Oct 11, 2024
@curquiza curquiza requested a review from ellnix October 14, 2024 09:56
@Nymuxyzo Nymuxyzo force-pushed the feat/federated-search branch from b2749b1 to d5f6edd Compare October 14, 2024 16:18
Copy link
Collaborator

@ellnix ellnix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution! Good job on the documentation comment, we should add those for the entire gem.

A few things regarding git:

  • Try to write commit messages in imperative sentences
    Federated search added
    Add federated search ✔️
  • Use git rebase to clean up the git history.

If you are new to git or unsure how to do this, feel free to reach out to me.

@@ -16,4 +16,19 @@
expect(response['results'][0]['estimatedTotalHits']).to eq(0)
expect(response['results'][1]['estimatedTotalHits']).to eq(0)
end

it 'does a federated search with two different indexes' do
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While this test is sufficient to see that the server has indeed understood that this is a federated search and not a regular multi-search, there should be separate tests to see if the server returns what you would expect under these circumstances.

In addition we should check that limit and offset are sent properly and work as expected.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does that meet your expectations?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, from what I understood here @Nymuxyzo you would need to make two tests:

  • one asserting the inexistence of the presence of the param federation in the response
  • one asserting that you explicitly called the multi_search with a federation request, since the doc says a non-null value the value could be multi_search(data, federation: true, federation_options: {....}) # where federation options are not required

https://www.meilisearch.com/docs/learn/multi_search/multi_search_vs_federated_search#what-is-federated-search

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@brunoocasali I'm sorry, but I don't understand the statement. When I look at the added test in the PHP SDK (tests/Endpoints/MultiSearchTest.php), I recognize my pattern to a very large extent.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would very much like to bring this issue to a good end.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've also read and reread

one asserting the inexistence of the presence of the param federation in the response
one asserting that you explicitly called the multi_search with a federation request, since the doc says a non-null value the value could be multi_search(data, federation: true, federation_options: {....}) # where federation options are not required

and I'm sorry @brunoocasali I can't make sense of it xD

I think this might be a typo:

one asserting the inexistence of or the presence of the param federation in the response

but the presence of _federation is already looked for in the test.

Meanwhile this:

multi_search(data, federation: true, federation_options: {....}) # where federation options are not required

I'm not sure I'm following with the distinction between federation and federation_options, there is only one federation option, which can either be an object of options or a truthy value to enable it with defaults.

In any case, this should work fine as long as we don't mess with it, ruby is dynamically typed and there is nothing preventing someone from passing a boolean as federation.

Copy link
Member

@brunoocasali brunoocasali Jan 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, @ellnix, I agree with your previous message; I will point out the change in another comment.

@Nymuxyzo Nymuxyzo force-pushed the feat/federated-search branch 10 times, most recently from da76a89 to 77ed25b Compare October 30, 2024 00:48
@Nymuxyzo
Copy link
Contributor Author

Thanks for the contribution! Good job on the documentation comment, we should add those for the entire gem.

A few things regarding git:

* Try to write commit messages in [imperative sentences](https://www.grammarly.com/blog/sentences/imperative-sentences/)
  `Federated search added` ❌
  `Add federated search` ✔️

* Use git rebase to clean up the git history.

If you are new to git or unsure how to do this, feel free to reach out to me.

Commit message rewritten and git rebase executed

@Nymuxyzo Nymuxyzo requested a review from ellnix October 30, 2024 22:30
@curquiza curquiza changed the title Federated search added (#558) Federated search added Nov 6, 2024
@Nymuxyzo Nymuxyzo force-pushed the feat/federated-search branch from 77ed25b to 83a79c8 Compare November 6, 2024 20:44
brunoocasali

This comment was marked as outdated.

@brunoocasali brunoocasali added the enhancement New feature or request label Dec 11, 2024
@brunoocasali brunoocasali self-requested a review December 11, 2024 12:10
@@ -16,4 +16,19 @@
expect(response['results'][0]['estimatedTotalHits']).to eq(0)
expect(response['results'][1]['estimatedTotalHits']).to eq(0)
end

it 'does a federated search with two different indexes' do
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, from what I understood here @Nymuxyzo you would need to make two tests:

  • one asserting the inexistence of the presence of the param federation in the response
  • one asserting that you explicitly called the multi_search with a federation request, since the doc says a non-null value the value could be multi_search(data, federation: true, federation_options: {....}) # where federation options are not required

https://www.meilisearch.com/docs/learn/multi_search/multi_search_vs_federated_search#what-is-federated-search

@Nymuxyzo Nymuxyzo force-pushed the feat/federated-search branch from 8e87707 to 37bb760 Compare December 14, 2024 14:36
@brunoocasali
Copy link
Member

@ellnix can you make the last test to ensure the federation is working?

@Nymuxyzo Nymuxyzo force-pushed the feat/federated-search branch from 37bb760 to 7e15ead Compare January 8, 2025 18:44
@curquiza curquiza requested a review from brunoocasali January 10, 2025 15:56
@curquiza curquiza force-pushed the feat/federated-search branch from 7e15ead to 62ccd9d Compare January 10, 2025 16:00
@Nymuxyzo Nymuxyzo force-pushed the feat/federated-search branch from 62ccd9d to 6b51188 Compare January 17, 2025 23:45
Copy link
Collaborator

@ellnix ellnix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I apologize for taking so long to get to this, I spent quite a lot of time reading and rereading what happened in this thread and the reference in the meilisearch documentation.

There is also a code sample that the issue of this PR does not mention for some reason (this example is from meilisearch-js):

multi_search_federated_1: |-
  client.multiSearch({
    federation: {},
    queries: [
      {
        indexUid: 'movies',
        q: 'batman',
      },
      {
        indexUid: 'comics',
        q: 'batman',
      },
    ]
  })

if you would like to add it, it's not a big deal if you would like to focus strictly on the code.

Please don't hesitate to reach out to me if I said something stupid (chances are I did). I'm also on the meilisearch discord server under the same name, feel free to add me 😄

# @param [Hash] federation_options
# - `limit`: number of results in the merged list
# - `offset`: number of results to skip in the merged list
def multi_search(data, federation_options = nil)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not a fan personally of two positional arguments with almost no explanation as to what each of them does in the non-existent documentation. Especially since in the reference, federation is listed first (I'm guessing it's alphabetical)

I would prefer if we took the js approach and used query: and federation: keyword arguments.

This would mean breaking code that was written to use multi_search before this PR. But, we can avoid that if we have one data=nil positional argument which we take as a query, and warn the user on its use:

def foo(bad_arg=nil, query: {}, federation: nil)
  if bad_arg
    puts "Bad arg is bad!"
  end
  
  query.update(bad_arg)
  
  {
    query: query,
    federation: federation
  }
end

foo({ q: 'hello world'}, federation: true)
# Bad arg is bad!
# => {:query=>{:q=>"hello world"}, :federation=>true}

Obviously, please don't use puts (Utils.soft_deprecate! might be appropriate) to throw warnings, and name the arguments appropriately if you choose to do this.

@@ -16,4 +16,19 @@
expect(response['results'][0]['estimatedTotalHits']).to eq(0)
expect(response['results'][1]['estimatedTotalHits']).to eq(0)
end

it 'does a federated search with two different indexes' do
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've also read and reread

one asserting the inexistence of the presence of the param federation in the response
one asserting that you explicitly called the multi_search with a federation request, since the doc says a non-null value the value could be multi_search(data, federation: true, federation_options: {....}) # where federation options are not required

and I'm sorry @brunoocasali I can't make sense of it xD

I think this might be a typo:

one asserting the inexistence of or the presence of the param federation in the response

but the presence of _federation is already looked for in the test.

Meanwhile this:

multi_search(data, federation: true, federation_options: {....}) # where federation options are not required

I'm not sure I'm following with the distinction between federation and federation_options, there is only one federation option, which can either be an object of options or a truthy value to enable it with defaults.

In any case, this should work fine as long as we don't mess with it, ruby is dynamically typed and there is nothing preventing someone from passing a boolean as federation.

body = Utils.transform_attributes(data)

http_post '/multi-search', queries: body
http_post '/multi-search', queries: body, federation: federation_options
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

federation_options should be transformed to camel case with Utils.transform_attributes.

To be clear, it wasn't necessary when you first started this PR, but now there are 2 new options (facetsByIndex and mergeFacets) which won't work if you pass them as snake_cased symbols.

Comment on lines +40 to +41
limit: 20,
offset: 0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice if these values were not the same as the defaults, but it's not a big problem.

# @param [Hash] federation_options
# - `limit`: number of results in the merged list
# - `offset`: number of results to skip in the merged list
def multi_search(data, federation_options = nil)
Copy link
Member

@brunoocasali brunoocasali Jan 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, basically what @ellnix suggested is to do something like:

Suggested change
def multi_search(data, federation_options = nil)
def multi_search(data, federation: nil)

and I agree with that :)

@ellnix ellnix mentioned this pull request Jan 24, 2025
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[v1.10.0] Federated search
3 participants