Cpu issue

[team-boo]

Due diligence

• I have done my due diligence in trying to find the answer myself.

Topic

The PyTorch implementation

Question

python -m moshi.server --device cpu --hf-repo kyutai/moshika-pytorch-bf16

doesn't work .

OS Debian 120+ core cpu 40GB RAM

I have always

"RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx"

[LaurentMazare]

Sorry the pytorch version requires an nvidia gpu to work (contrary to text models, a speech model like moshi needs to be real time and this would be very tricky on cpu only even with 120 cores).

[team-boo]

There is any version that can work without nnvidia gpu?

i can give 200 cores if needed.

[LaurentMazare]

You can try the rust version, everything should be cpu compatible but it's unlikely anyone tried this out so you may well run into issues (on top of the model likely not being real-time).

[team-boo]

Thnx.

[team-boo]

after python -m moshi.server --device cpu --hf-repo kyutai/moshiko-candle-bf16
again error

Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/home/llll/myenv/src/moshi/moshi/moshi/server.py", line 275, in
main()
File "/home/llll/myenv/src/moshi/moshi/moshi/server.py", line 225, in main
lm = loaders.get_moshi_lm(args.moshi_weight, args.device)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/llll/myenv/src/moshi/moshi/moshi/models/loaders.py", line 152, in get_moshi_lm
load_model(model, filename)
File "/home/llll/myenv/lib/python3.11/site-packages/safetensors/torch.py", line 221, in load_model
raise RuntimeError(error)
RuntimeError: Error(s) in loading state_dict for LMModel:
Missing key(s) in state_dict: "depformer.layers.0.gating.0.linear_in.weight"

[rayfiyo]

Sorry if this is totally misguided.
In summary, I believe this issue is caused by a mismatch between the model checkpoint
and the current Moshi code (specifically missing parameters in depformer/gating).
It appears to be beyond the scope of the kyutai-labs/moshi repository itself, since updating
the checkpoint or reverting code versions would be required to resolve it.

Therefore, I recommend closing this issue here.
Any necessary updates likely need to happen on the model side or in a fork that aligns
with the older model format.

after python -m moshi.server --device cpu --hf-repo kyutai/moshiko-candle-bf16
again error

Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/home/llll/myenv/src/moshi/moshi/moshi/server.py", line 275, in
main()
File "/home/llll/myenv/src/moshi/moshi/moshi/server.py", line 225, in main
lm = loaders.get_moshi_lm(args.moshi_weight, args.device)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/llll/myenv/src/moshi/moshi/moshi/models/loaders.py", line 152, in get_moshi_lm
load_model(model, filename)
File "/home/llll/myenv/lib/python3.11/site-packages/safetensors/torch.py", line 221, in load_model
raise RuntimeError(error)
RuntimeError: Error(s) in loading state_dict for LMModel:
Missing key(s) in state_dict: "depformer.layers.0.gating.0.linear_in.weight"

[LaurentMazare]

The kyutai/moshiko-candle-bf16 contains the candle/rust version of the weights, the weights to be used with pytorch are in kyutai/moshiko-pytorch-bf16.