A single pretrained model estimates causal effects on new data without retraining.

Figuring out whether a treatment or action actually causes an effect from data usually takes a lot of custom work. For every new problem, researchers must suggest a causal mechanism, pick a compatible tool to estimate it, and train that model from scratch. To fix this slow process, researchers introduce causal foundation models. In regular machine learning, foundation models are large neural networks trained once at scale and used on new tasks without retraining. Causal foundation models apply that same idea to causal questions. They use in-context learning to estimate causal effects, like the average effect of a treatment, on completely new datasets without any model updates. This work provides a practical guide to the topic, complete with background concepts, code examples, and Jupyter notebooks.

Why it matters

Causal inference is used in fields like healthcare and economics to understand what works. Currently, each problem requires building a new model, which is slow and needs experts. Causal foundation models could allow quicker, easier analysis across different problems without starting from scratch each time.

Questions answered