Original title: A Privacy Budgeting Framework for Online Experimentation
Gilian R. Ponte, Alina Ferecatu · August 2026
Websites run experiments to personalize ads and banners for you. But outside observers can watch what you see and guess personal details about you. Researchers created a privacy risk budgetA limit set on how much private information can be leaked through the experiment. to limit how much an outside party can learn, using a mathematical tool called differential privacyA technique that adds small noise to data so that the results don't reveal too much about any single person.. Websites can set this budget ahead of time. The researchers tested two ways to spend it across visitors: a constant strategy that spends the same risk, and a dynamic strategy that changes spending across visitors. They tested both in website design and recommendation systems. They found that the dynamic strategy works best in longer and more complex experiments. Also, sharing a total privacy budget smartly across all of a company's experiments greatly improves learning performance.
Imagine you are a chef trying out new recipes on customers. You serve different dishes to different tables to see which is most popular. But a rival chef watches which dishes you bring and learns your customers' preferences – maybe which tables order expensive wine. To prevent this, you set a limit: no more than one new dish per table, or vary the timing so the rival cannot keep track. In this paper, the firm sets a privacy budget on how much a third party can learn from seeing which ads or recommendations a visitor gets, and chooses whether to spend it evenly or adjust it per visitor, especially useful in longer experiments.
If you browse a website and get different ads and recommendations, the site is likely running an experiment to learn what you like. Without safeguards, any third party watching could figure out your general taste or demographic from what you are shown. This framework gives companies a way to set a hard limit on how much information leaks from their experiments, so they can keep learning about their users without exposing them.
The framework assumes firms can accurately set a privacy budget ex ante and that third-party inference is limited to observing which banners or recommendations are shown. It does not address other side channels or more sophisticated adversaries. The empirical validation uses a specific set of 78 experiments, and the performance gains may not generalize to all experimental settings.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
A Privacy Budgeting Framework for Online Experimentation — Gilian R. Ponte, Alina Ferecatu
Abstract licence: CC0
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.