Original title: mzCache: On-Device LLM Memory Management under Multitasking
Hongseung Yu, Minsung Kim, Jongseok Park, Kyunghan Lee · September 2026
Running smart AI models directly on mobile phones is tricky. When you jump between different apps, the phone runs low on memory. The operating system kicks the AI out of fast memory, dumping its stored data and weights. When you use the AI again, the phone must slowly reload everything from storage or redo huge calculations. This makes the model pause before answering. Researchers built mzCache to fix this. It cuts the AI memory into tiny pieces so it can smoothly dump and reload small chunks. mzCache uses the phone's shared memory chips to let the graphics processor answer right away while the main processor quietly reloads the rest in the background. It speeds up the first response by 2.1-5.5$\times$.
Imagine cooking several dishes at once in a small kitchen with limited counter space. Every time you switch to a new dish, you must put away the ingredients from the previous one, and getting them back out later is slow because they're stored in distant cabinets. In this paper, the AI's memory is like those ingredients, and mzCache manages them in fine-grained sections that can be partially restored without full retrieval, using the phone's shared memory to avoid delays.
For phone users, AI features like chatbots slow down when switching apps because memory is cleared. mzCache keeps these AI tools fast and smooth during multitasking, improving everyday mobile experiences.
The abstract doesn't detail specific limitations, but the system is implemented only on Android using llama.cpp and evaluated in particular multitasking scenarios. It may not generalize to other platforms or inference frameworks.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
mzCache: On-Device LLM Memory Management under Multitasking — Hongseung Yu, Minsung Kim, Jongseok Park et al.
Abstract licence: CC0
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.