What are the recommended requirements for LocalAI

[El-Tommy]

Is there any specification of minimum requirements or recommendations for using LocalAI?

We are planning to use LocalAI to generate embeddings and as a chat model alongside Flowise AI.

From what I saw in the documentation, the biggest recommendation is to use SSD instead of HDD.

But at the hardware level like CPU and RAM I didn't find anything.

[jtwolfe]

This depends highly on the model used and weather or not you want to use the host system at the same time as doing inference.

While LocalAI is a small program the memory consumption is basically the same size as the model as it needs to load the whole thing into memory to be able to operation quickly, this goes for GPU also.
I have found that you can get away with running smaller models on systems that have very low ram on windows as long as you have a VERY fast ssd and I mean like 7Gb/s w/r m.2.

CPU compute - throw as many cores as you can at it.
GPU compute - 12-16G cards made after 2018 will get you decent performance

Limits - you will never be able to load the monster models unless you rent GPUs or spend 90kUSD on chips