LLM judges reuse the base model's structure, with fine-tuning adding just two adjustments.

People often use large language models as judges to score written summaries, but no one really knows how they decide on a rating. Researchers looked inside these judge models to see how they work. They made clean and corrupted summary pairs with specific errors in readability and adequacy. Then, they tested two judge models called Themis and Prometheus by tracking their inner signals. They found that both models use a clear two-step process. In the lower layers, below layer 15, the model uses attention to spot local mistakes and pass that evidence to the very last word position. In the higher layers, internal processing networks combine these signals and choose the final score. The final decision sets in sharply at layer 26 in Themis and layer 25 in Prometheus. Fine-tuning adjusts existing parts of the model rather than creating this scoring system from scratch.

Why it matters

LLM evaluators are widely used to score AI-generated text and train models. Understanding their internal mechanisms is important for making them more reliable and transparent in practice.

Questions answered