Add SIMLAYERKV

[dame-cell]

Feature

SimLayerKV dynamically identifies lazy layers in LLMs—layers that focus primarily on initial and recent tokens allowing selective KV cache trimming. This reduces memory usage during inference without requiring additional training, making it more efficient and adaptive than static methods.

Paper

SimLayerKV
github

not sure if this has been added but if not I would like to work on it

[maxjeblick]

Thanks a lot for opening this issue, this method definitely looks relevant!

I would like to work on it

That would be great! Feel free to ask any questions in this thread. You can also open a draft PR to discuss the current status of the implementation.

[dame-cell]

awesome I have actually been working on it might open a draft by tomorrow :)
Some questions:

• Do i need to write a custom tests or just use the existing ones ?

[maxjeblick]

Do i need to write a custom tests or just use the existing ones ?

In general, any new press should be added here. Apart from that, additional tests are always helpful :)