feat(mlx_lm): basic speculative decoding support in mlx_lm.generate / mlx_lm.server#954

Closed

llllvvuu wants to merge 3 commits intoml-explore:mainfrom llllvvuu:feat/speculative_decoding

+264-119

Commits on Aug 21, 2024

feat: support batch input in generate()
llllvvuu
committed

Commits on Aug 23, 2024

feat: show batch generation progress
llllvvuu
committed

Commits on Aug 27, 2024

feat: basic speculative decoding support in mlx_lm.generate / mlx_lm.server
llllvvuu
committed