Original title: CORE: Improving Compositional Reasoning in MLLM Embedding via Reranker Distillation
Tingyu Song, Mingxin Li, Yanzhao Zhang, Dingkun Long +4 · September 2026
Modern vision-and-language AI models create embeddings, which are numerical summaries of images and text used for searching. However, they struggle to tell the difference between scenes that have the same objects mixed up in different ways, like a red car and blue bike versus a blue car and red bike. Interestingly, the same AI can tell these apart when run as a slower, more detailed checker called a reranker. Researchers created CORE to transfer this skill into the faster embedding model. CORE builds lists of examples with five matching levels and teaches the embedder to copy the reranker's rankings. This training helped their embedding model outperform competitors on test challenges while keeping standard search skills strong.
Imagine packing a suitcase for a trip where you don't know the daily activities. If you pack all at once, you might miss items that become important later. In this paper, the embedding model is like packing once upfront, often missing details. The reranker is like checking and reorganizing items each day as plans change. The training method teaches the packing model to learn from the daily reordering, so it packs better from the start.
This helps search and recommendation systems understand subtle differences in queries, like distinguishing 'dog chasing cat' from 'cat chasing dog,' making results more accurate for users.
The abstract does not explicitly state limitations, but the work is validated on specific benchmarks like COLA, SUGARCREPE++, and NEGBENCH, and focuses on five compositional matching levels.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
CORE: Improving Compositional Reasoning in MLLM Embedding via Reranker Distillation — Tingyu Song, Mingxin Li, Yanzhao Zhang et al.
Abstract licence: CC0
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.