Model size stops predicting AI answer quality once adapted for factory use.

Factory workers need conversational computer assistants to read machine manuals on site, but smart models are usually too large for local factory hardware. Researchers wanted to shrink these models without losing accuracy. They discovered that after compressing a model and training it to use retrieved documents, model size stops predicting how well it answers questions. While general skill drops almost linearly as parameters decrease, specific answer quality does not. To solve this, they built one large supernetwork that contains many smaller sub-networks. This lets them pick the single best sub-network for each specific machine based on tested speed and quality. In a factory test, their method recovered two thirds of lost quality across three hardware tiers using 1.3 to 5 watts standby.

Why it matters

Factory workers can get reliable AI assistance without needing expensive hardware. This makes advanced technology accessible on shop floors where resources are tight.

Questions answered