Swap chat model provider at runtime

[ikwattro]

More of a question than an issue first as nothing is really mentioned in the docs.

Version : 0.24.0.CR1

With the following settings :

# langchain4j
quarkus.langchain4j.log-requests=true
quarkus.langchain4j.log-responses=true
quarkus.langchain4j.chat-memory.memory-window.max-messages=20

# openai
quarkus.langchain4j.m1.chat-model.provider=${M1_CHAT_MODEL_PROVIDER:ollama}
quarkus.langchain4j.openai.m1.api-key=${OPENAI_API_KEY:''}

# ollama
quarkus.langchain4j.ollama.base-url=${OLLAMA_BASE_URL:http://localhost:11434}

I was hoping to be able to switch between ollama and openai with the same build depending if I'm running locally or in deployed environment, but it seems that even when then M1_CHAT_MODEL_PROVIDER=openai variable is passed to the container ( as in docker container ), the app continues trying to use ollama.

Did I miss something or is the model provider constrained to be set at build time ? If it is the case, what is the recommended way to approach this ?

[geoand]

The model to use is indeed constrained to build time

[ikwattro]

Thanks @geoand , what is the recommendation on how to approach this ? Can I somehow register all of them with different names and choose one based on another config property ?

[geoand]

It depends on what exactly you want to achieve.

I believe @maxandersen had some example were was doing something similar