Original title: Masked Autoregressive Speech Enhancement with Continuous Neural Audio Codec Representations
Yoto Fujita, Simon Leglaive, Laurent Girin · September 2026
Background noise can make recorded speech hard to understand. Many speech enhancement tools clean up audio by breaking it into discrete tokens, which act like digital building blocks. However, continuous representations—smooth mathematical descriptions of sound—can produce clearer, more natural speech. Researchers created MARSE, a method that cleans up noisy recordings using these continuous representations. MARSE works by hiding parts of the clean speech and using an iterative decoding process, which means it guesses and rebuilds the missing clean pieces step by step. To test it, the team compared several decoding policiesStrategies that determine how the model fills in missing data during the generation process. while keeping the neural network and training conditions identical. They found that MARSE lets users flexibly balance speech enhancement quality against the computational cost required to run it.
Imagine you're finishing a jigsaw puzzle, but some pieces are missing. Normally, you might guess based on the shape and nearby colors. Now suppose the pieces have smooth color gradients instead of fixed patterns, and you fill them in one at a time, using the surrounding pieces as a guide. In this work, the system hides parts of the audio and uses continuous values to fill them in step by step, which lets it balance how much effort to use for accuracy.
This matters for people using voice technology in noisy settings, like phone calls or hearing aids. It allows systems to be tuned for clearer audio or lower power use, depending on the situation.
The study uses a fixed Conformer model and DAC codec, so results may not directly apply to other architectures. The abstract does not address how the method handles varying noise conditions or real-time deployment challenges.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
Masked Autoregressive Speech Enhancement with Continuous Neural Audio Codec Representations — Yoto Fujita, Simon Leglaive, Laurent Girin
Abstract licence: CC0
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.