Skip to content

Commit

Permalink
Add KV Cache for CPU Runtime
Browse files Browse the repository at this point in the history 
This PR introduces KV Cache support for the CPU runtime:
* Implementation of KV Cache TIR for CPU-based processing.
* Updates to the relevant runtime components to integrate KV Cache.

Co-authored-by: HMZ <[email protected]>
Co-authored-by: ShouChenChiu <[email protected]>
  • Loading branch information
@mengshyu @ShouChenChiu
3 people committed Jan 29, 2025
1 parent 2340081 commit 135bcaf
Show file tree
Hide file tree
Showing 3 changed files with 1,320 additions and 15 deletions.
Loading

0 comments on commit 135bcaf

Please sign in to comment.