[v1.10.0] AI-powered search changes

[brunoocasali]

Following this central issue

Explanation of the feature

• Rest embedder
  • Removed parameters: query, inputField, inputType, pathToEmbeddings and embeddingObject.
  • Replaced by request and response
  • New parameter: headers
• Add url parameter to the OpenAI embedder
• dimensions is now available as an optional parameter for ollama embedders.

Usage: https://meilisearch.notion.site/v1-10-AI-search-changes-737c9d7d010d4dd685582bf5dab579e2?pvs=74

TODO

• Rest embedder
  • Remove the following parameters: query, inputField, inputType, pathToEmbeddings and embeddingObject.
  • Add the new parameters request, response and headers (cf usage for more information)
• Add url parameter to the OpenAI embedder
• Add dimensions for Ollama embedders.
• Update the tests accordingly

[ellnix]

I don't see that anything needs to be done here. In fact, this gem does not have any real support for AI search at all.

If you like, we could create convenience methods for setting embedders, but until we do the only way to set them is Index#update_settings.

Should I open an issue for this?

[adenta]

@brunoocasali something needs to happen here, your documentation points to methods that don't exist

client.index('INDEX_NAME').update_embedders(
default: {
source: 'openAi',
api_key: 'anOpenAiApiKey',
model: 'text-embedding-3-small',
document_template: "A document titled '{{doc.title}}' whose description starts with {{doc.overview|truncatewords: 20}}"
}
)

[adenta]

https://www.meilisearch.com/docs/reference/api/settings?utm_campaign=vector-search&utm_source=blog&utm_medium=ai-react#embedders-experimental

[adenta]

or wait, does @ellnix also work for MS?

[adenta]

it doesn't appear that client.index('Post').set_settings is real either :/

irb(main):005> client.index('Post').set_settings
(irb):5:in `<main>': undefined method `set_settings' for #<MeiliSearch::Index:0x000000011d8f70a0 @uid="Post", @primary_key=nil, @base_url="http://127.0.0.1:7700", @api_key=nil, @options={:timeout=>1, :max_retries=>0, :convert_body?=>true}, @headers={"Content-Type"=>"application/json", "User-Agent"=>"Meilisearch Ruby (v0.28.4)"}> (NoMethodError)

client.index('Post').set_settings
                    ^^^^^^^^^^^^^
Did you mean?  get_settings
               reset_settings

(irb):6:in `<main>': undefined method `set_settings' for #<MeiliSearch::Rails::SafeIndex:0x000000011a859588 @raise_on_failure=true, @index=#<MeiliSearch::Index:0x000000012e5d5870 @uid="Post", @primary_key=nil, @base_url="http://localhost:7700", @api_key="RXBbmZbF6bof9tY0elQBTI7y1aQxWK3GemYyYgb_84s", @options={:timeout=>5, :max_retries=>0, :convert_body?=>true, :client_agents=>"Meilisearch Rails (v0.14.2)"}, @headers={"Content-Type"=>"application/json", "Authorization"=>"Bearer RXBbmZbF6bof9tY0elQBTI7y1aQxWK3GemYyYgb_84s", "User-Agent"=>"Meilisearch Rails (v0.14.2);Meilisearch Ruby (v0.28.4)"}>> (NoMethodError)

Post.index.set_settings
          ^^^^^^^^^^^^^
Did you mean?  get_settings
               reset_settings
irb(main):007> Post.index.set_settings

[adenta]

looks like update_settings is real

[ellnix]

your documentation points to methods that don't exist

Unfortunately, the documentation website seems to work off of the main branch, while that method is implemented, it was part of another issue #590 and implemented in #592. Those changes have not been part of a release as of yet.

This should probably change, but I'm not aware of anything I can do on my end to fix it. The SDKs repos are set up in very particular ways with the bors (the meili bot) and I cannot change the branches or the structure.

or wait, does @ellnix also work for MS?

I do not, I just maintain this gem (and the rails one) along with Bruno, who works for MS and coordinates the work on the various SDKs.

[ellnix]

it doesn't appear that client.index('Post').set_settings is real either :/

I do not see that in the documentation anywhere, please link it for me and I'll make a PR to update the syntax.

Edit: I see that I said it in my above message, apologies, that was just something I wrote quickly. The correct method is indeed update_settings. I edited my comment.

[adenta]

@ellnix so who can make a release?

Also, can you show me an example of how to create an embedder with the update_settings method?

I can put together a blog post documenting how to get semantic search running locally if so

[ellnix]

@ellnix so who can make a release?

I can trigger it, but I would need my PR that changes the version number to be approved by a member of MS, since I don't have write access to the repository.

That PR is just a convenience method though, you can access the full capability of MS without it.

Also, can you show me an example of how to create an embedder with the update_settings method?

You would simply have to

client.index(INDEX_UID).update_settings({
  embedders: {
    # embedder configuration goes here
  }
})

So the example in the documentation would become:

client.index(INDEX_UID).update_settings({
  embedders: {
    default: {
      source:  'openAi',
      api_key: 'anOpenAiApiKey',
      model: 'text-embedding-3-small',
      document_template: "A document titled '{{doc.title}}' whose description starts with {{doc.overview|truncatewords: 20}}"
    }
  }
})

That is all the ruby-specific knowledge you need. You can then get started with https://www.meilisearch.com/docs/learn/ai_powered_search/getting_started_with_ai_search

In the ruby gem, the option keys are passed as snake_cased symbols, but you can use camelCased strings/symbols (that most of the documentation uses) if that confuses you. You'll get a warning but everything will be fine.

[ellnix]

@adenta

Also, you would need to enable vector store currently, since it is considered an experimental feature in the latest 1.12.6 version of MS (it's planned to be stable in 1.13 as far as I know).

You are probably tired of hearing it, but the method to change experimental settings is currently an open PR waiting for a review #594

If you are using Meilisearch Cloud, you could use the dashboard, or if you are hosting MS elsewhere you can simply run the curl example in the documentation once and it will be enabled:
https://www.meilisearch.com/docs/reference/api/experimental_features#configure-experimental-features

curl \
  -X PATCH 'MEILISEARCH_URL/experimental-features/' \
  -H 'Content-Type: application/json'  \
  --data-binary '{
    "vectorStore": true
  }'