Original title: From Reweighting to Rewriting: Unlocking the Intervention Effects of Influential Samples in Training Data Attribution
Yuzhang Luo, Chenpeng Wang, Jianhui Chen, Liangming Pan · September 2026
Researchers want to know which training data points shape how an AI behaves. A common tool called influence functions spots these key examples by estimating what happens if you change their importance weights. But changing weights often does little more than picking data at random. This makes scientists wonder: are these examples actually useless, or is changing weights the wrong tool? To fix this, researchers kept the original instructions but rewrote the answers to either support or oppose the target behavior. They tested this on four open-weight models. Rewriting produced stronger, longer-lasting, and bidirectional shifts in behavior, while reweighting was weak and inconsistent. Influence-selected examples gave more leverage for rewriting than other selection methods, keeping changes focused on the target tasks.
Imagine you are baking a cake, but it turns out too salty. You could try adjusting the salt amount a little, but if the recipe instructions are confusing, that might not help much. In this paper, instead of tweaking ingredient quantities like adjusting weights, they change the baking instructions for the steps that were causing problems, which fixes the taste more effectively. Here, the key training examples are like the confusing recipe steps, and rewriting their responses is like correcting the instructions to guide the model better.
For AI developers trying to adjust model behavior, this means rewriting training data responses is far more effective than tweaking example weights. It could lead to safer and more reliable model updates, affecting anyone using AI systems.
The abstract does not specify if this applies to models other than open-weight LLMs or to behaviors beyond epistemic abstention and safety refusal. It focuses on response rewriting versus reweighting, not other intervention types, and the evaluation is limited to four models.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
From Reweighting to Rewriting: Unlocking the Intervention Effects of Influential Samples in Training Data Attribution — Yuzhang Luo, Chenpeng Wang, Jianhui Chen et al.
Abstract licence: CC0
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.