Combining prediction models helps initially, but benefits fade fast, leaving some problems unsolvable.

Knowledge graphsKnowledge graphsStructured databases of entities and relationships, used in web applications for tasks like search and recommendations. organize facts for web applications like search engines and recommender systems. Sometimes these graphs have missing facts. Computer models try to guess these missing connections, a task called link predictionlink predictionThe task of predicting missing connections or facts in knowledge graphs.. However, different models, or even the same model trained multiple times, often produce very different answers for the exact same question. The researchers wanted to see if these models learn different, useful pieces of information that could help each other. To test this, they used a theoretical system called an oracleoracleA theoretical selector that always chooses the best prediction from a set of models for each query, serving as a performance ceiling.. For every single query, this oracle simply picks the best answer from a group of models, showing the highest possible score a combination could get. Across several architectures and benchmarksbenchmarksStandard datasets used to evaluate and compare the performance of various models., they found a big performance gap between single models and the oracle, proving models do hold complementary knowledge. However, this benefit quickly levels off as you add more models, leaving a stubborn group of queries that none of them can solve.

Why it matters

Web apps like search engines and recommendation systems rely on predicting missing links in knowledge graphs. Knowing how models combine can make these apps more accurate, but this paper reveals a ceiling to improvement with current methods.

Questions answered