A new dataset lets AI compare banks across Europe and the U.S. using reports averaging 198,000 words.

Comparing financial reports from different banks is tough for computers. These files are massive, technical, and use different formats. To test how well AI answers questions about them, researchers built a new benchmark called FinRAG-QA. They tested a system that looks up facts before writing answers, a method called retrieval-augmented generation. First, the system enriches snippets of text with helpful context and uses a specialized model to find the right information. This step greatly improves search quality. Then, a generator that focuses on reasoning writes the answer. If it receives the right facts, its accuracy jumps from 44.6% to 79.0%, though it runs roughly 20x slower. Giving the system just one top-ranked snippet worked better than feeding it larger chunks of text.

Why it matters

For investors and regulators trying to compare banks, automated tools struggle with the complexity and length of financial reports. This dataset makes it possible to build AI that accurately answers questions across different banks, improving decisions and transparency.

Questions answered