Papers with Code paper Jun 26

ReFreeKV: Towards Threshold-Free KV Cache Compression

To reduce memory consumption during LLM inference, a handful of methods have been proposed for KV cache pruning. While these techniques can accomplish lossless memory reduction on ...

Papers with Code paper Jun 26

Simplified Sparse Attention via Gist Tokens

Sparse attention can reduce the cost of long-context inference, but most variants introduce new architectural components. We introduce Simplified Sparse Attention (SSA), a simpler ...