Privacy and security defenses in shared AI training can undermine each other.

Banks and hospitals want to train shared computer models without pooling private records in one place. Federated learning lets them do this by sharing model updates instead of raw data. However, two big problems remain. Sneaky attacks can reconstruct private files from these updates, and dishonest participants can send corrupted updates to sabotage the shared model. To solve this, researchers built a framework called DP-BR-FedAvg. It adds mathematical noise to updates so individual records stay private. It also filters out extreme updates by trimming away suspicious values before averaging them together. The researchers tested it against saboteurs. Standard averaging failed completely on rare, important cases, scoring an F1-score of 0.030. The new framework held up much better, reaching an F1-score of 0.119 while keeping data private. The team also discovered that adding privacy noise slightly hurts defenses against bad updates, showing that privacy and defense push back against each other.

Why it matters

Banks and hospitals must train AI models together without sharing sensitive data, but current methods either leak private information or get hijacked by bad actors. This framework lets them collaborate while protecting privacy and resisting attacks, which is essential for tasks like fraud detection and medical risk scoring.

Questions answered