A system compresses its memory cache 128 times without losing any information, using a signal it already has.

AI models use a memory called a KV cacheKV cacheA memory storage in AI models that holds key and value information from previous tokens to avoid recomputation. to remember past text. Compressing this memory early is hard because the model does not know what future questions will ask. Old compression tricks fail here, scoring between 0.00 and 0.33 on memory tests because they rely on clues that have not happened yet. To fix this, researchers introduced VestigeKV for a model called Kimi Linear. The system does not discard tokens or need new training. Instead, it reads a tiny leftover 64-dimensional channel already inside the memory. This channel reveals token importance on its own. VestigeKV reads 11% of each row to keep the best rows in an active memory area while moving the rest into a bit-exact archive. This archive stays reachable whenever needed. Retrieval accuracy reaches 1.00 under 8x compression and 0.92 under 32x across 8k to 65k context lengths.

Why it matters

This allows AI models to handle much longer texts, like entire books or lengthy conversations, without forgetting important details. It improves efficiency in applications such as document summarization or chatbots that need to maintain context over long interactions.

Questions answered