Original title: Can LLMs Discover Scientific Laws in Real and Parallel Worlds?
Yiming Huang, Ziche Liu, Zhuohang Wu, Yiqian Wang +10 · September 2026
Scientists use data to discover mathematical equations that describe nature. Researchers wanted to know if large language models can do this, or if they just repeat equations they already memorized. To test this, they built a benchmark called SCILAWS-BENCH. They gathered real data and papers, creating two tests. In the first test, models look at fixed real data and must propose equations that fit new data and make scientific sense. In the second test, models actively run tests in simulated worlds to uncover hidden, synthesized equations. The researchers discovered three big things. First, an equation that fits the numbers well does not always make real scientific sense. Second, memorizationWhen LLMs reproduce known formulas from their training data instead of creating new ones. heavily shapes whether models simply copy familiar formulas or go beyond them. Third, testing multiple guesses in a best-of-N study revealed a selection bottleneckA challenge in choosing the best law from multiple options generated by the AI., meaning it is hard to pick the best equation from many attempts.
Imagine you're trying to learn a new recipe by heart. You might memorize the steps and get a dish that tastes similar, but without understanding why certain ingredients are used, you can't adapt it if you need to change something. In this paper, LLMs can mimic the steps of scientific laws to predict data, but they often lack the deep understanding needed for genuine discovery.
If AI can't truly discover scientific laws, scientists might waste time on misleading results. This benchmark helps assess whether AI can be trusted for real scientific progress, affecting fields like medicine and climate research.
The abstract doesn't cover all scientific domains or types of LLMs. It focuses on evaluation rather than developing new discovery methods. The findings are specific to the problems and data used in SCILAWS-BENCH.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
Can LLMs Discover Scientific Laws in Real and Parallel Worlds? — Yiming Huang, Ziche Liu, Zhuohang Wu et al.
Abstract licence: CC0
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.