Original title: DKL: Decoupled Knowledge Learning for Instruction-Tuned Language Models
Kushagra Bhushan, Meghanadh Pulivarthi, Sai Krishna Reddy Sathi, Gaurav Pandey +6 · September 2026
Instruction following language models need new knowledge, but standard lookup methods fail if they pull up wrong or incomplete facts. This failure causes the models to make things up. Teaching new facts directly to the model can wipe out its ability to follow instructions, unless researchers run costly extra training. Generating huge sets of practice questions to avoid this takes too much work. Researchers solved this with Decoupled Knowledge Learning, or DKL. Instead of teaching the new facts to the instruction-trained model, they train the simpler base model on the raw text. Then, they merge the newly trained base model weights into the instruction following model. This process shares the new knowledge while keeping the instruction following skills intact. DKL lifted model accuracy from 54.17 to 79.26 on retrieval failure cases, beating older methods with substantially less training data.
Imagine you have a well-organized cookbook for making weeknight dinners. Your family asks you to add breakfast recipes, but if you rewrite the entire cookbook, you might accidentally change the dinner recipes and make them harder to follow. In this paper, instead of rewriting the cookbook, you create a separate booklet with just the breakfast recipes and then merge those pages into the original cookbook. Now the cookbook has new breakfast recipes without messing up the dinners.
Companies using AI assistants often need to update their knowledge with new information. Currently, this requires expensive retraining or risks the model forgetting how to help users. DKL allows knowledge updates without these costs, making AI systems cheaper and more reliable to maintain.
The abstract only shows results for improving RAG accuracy on retrieval failure cases, so it's unclear how well it works for other types of knowledge updates or non-RAG applications. It also requires having a corresponding base LLM and performing model merging, which might introduce complexity not detailed here.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
DKL: Decoupled Knowledge Learning for Instruction-Tuned Language Models — Kushagra Bhushan, Meghanadh Pulivarthi, Sai Krishna Reddy Sathi et al.
Abstract licence: CC0
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.