Merging the least important image tokens speeds up vision models without losing accuracy.

Small gadgets like phones need to classify images quickly without using too much computing power. Vision Transformers are modern artificial intelligence models that process pictures by slicing them into pieces called tokens. However, running them can be slow and heavy for these small devices. To fix this without rebuilding the models, researchers created a selective token merging technique. Their system spots which image tokens carry the least useful details and merges mostly those less important parts together. In tests, this strategy let models run faster while keeping their accuracy high, beating older merging methods. By focusing reduction on the least critical picture pieces, devices can identify images swiftly and accurately without wasting precious processing power.

Why it matters

Edge devices like phones and cameras have limited battery and processing power. This work shows how to make image-classification models run faster on such devices while keeping accuracy nearly unchanged, so on-device AI becomes more practical and cheaper to run.

Questions answered