CoFiE skips 80% of video frames before encoding, boosting speed and accuracy.

Smart computer models that watch streaming video need to answer questions fast. Most systems try to speed up by cutting out extra data only after they have already turned whole frames into complex computer tokens. Because processing entire frames is very slow, cutting pieces out later does not save enough time. Researchers created CoFiE to fix this with a two-step evidence search. First, a coarse filter drops repeated or boring frames before the visual system even looks at them, saving only distinctive video frames. Second, a fine filter picks out the remaining frames that match the user's question during the language model stage. CoFiE cut end-to-end delay by up to 2.54 times and beat existing methods, even when discarding up to 80% of evidence frames.

Why it matters

For developers building real-time video apps like live translation or surveillance, this means processing longer videos with less delay and better answers. It directly addresses the tight latency constraints that limit current systems.

Questions answered