When Synthetic Speech Is All You Have: Better Call GRPO

TL;DR

Same synthetic speech. One different objective. 45% of the error gone.

Privacy rules make real banking calls nearly impossible to collect, so ASR systems are adapted on synthetic (TTS) speech — but they are trained with supervised fine-tuning, which overfits the synthetic artifacts. We adapt an LLM-based ASR with GRPO, a critic-free RL method that rewards low-WER hypotheses, and recover most of the synthetic-to-real gap without any additional data.

−0%

relative WER vs. SFT on synthetic speech (SFT→GRPO, 36.71 → 20.21)

36.71

WER after SFT→GRPO on synthetic speech only (SFT alone: 36.71)

24.79

insertion rate after GRPO (SFT: 24.79) — the hallucination tail vanishes

0 h

of real speech capture ≈⅔ of the achievable gain when mixed in

The problem

Real speech is the one thing you can't have

Customer–agent calls in regulated domains like banking carry biometric and financial data. GDPR and the EU AI Act restrict how they can be stored, shared and reused — so the community substitutes text-to-speech audio generated from transcripts. The catch: synthetic speech stays acoustically mismatched with real recordings, and every prior attempt to close that gap stays inside supervised fine-tuning.

…but the acoustic mismatch survives

live waveforms — stylized

Prosody quirks, phonetic glitches and imperfect room-impulse simulation keep synthetic speech locally unreliable. Token-level SFT must explain every such region against the reference — and that is exactly where it breaks.

Method

SFT imitates tokens — GRPO rewards transcripts

We keep the backbone deliberately simple and frozen — a WavLM-Large encoder feeding Llama-3.2-1B-Instruct through a small trainable projector (SLAM-ASR recipe), with LoRA adapters (r=16, α=32) — so that any difference comes from the adaptation objective, not the architecture.

Supervised fine-tuning

$$\mathcal{L}_{\mathrm{SFT}}(\theta)=-\,\mathbb{E}_{(x,y^\star)}\Big[\sum_{t=1}^{T}\log \pi_\theta\big(y_t^\star \mid x,\,y_{<t}^\star\big)\Big]$$

Teacher-forced, token-level cross-entropy: the model must reproduce the reference token by token, even where the synthetic audio doesn't support it.

TTS artifact→ exact token demanded→ LM prior takes over→ insertion chain

absorbs local synthetic defects → confident hallucinations

GRPO — critic-free sequence-level RL

$$A_i=\frac{r_i-\operatorname{mean}(\{r_1,\dots,r_G\})}{\operatorname{std}(\{r_1,\dots,r_G\})+\varepsilon},\qquad r_i = 1-\mathrm{WER}(o_i,y^\star)$$

$$\mathcal{J}(\theta)=\mathbb{E}\Big[\tfrac{1}{G}\sum_{i=1}^{G}\tfrac{1}{|o_i|}\sum_{t}\min\big(\rho_{i,t}A_i,\;\operatorname{clip}(\rho_{i,t},1{-}\epsilon,1{+}\epsilon)A_i\big)\Big]-\beta\,\mathrm{KL}\big[\pi_\theta\,\|\,\pi_{\mathrm{ref}}\big]$$

Whole transcripts are scored against the metric that matters; hypotheses better than their own group get reinforced, worse ones suppressed. A KL leash to the SFT checkpoint (β = 0.04) keeps decoding stable. G = 4 rollouts, τ = 0.8 — no value network.

local artifacts can't dominate a sequence-level reward

Interactive — one GRPO step

rewards → group-relative advantages

reference y* : i'd like to block my card

rewards use r = 1 − WER against the reference

Results · DefinedAI banking (6.55 h test)

Same data, different objective: −45% WER

All models start from an SFT checkpoint trained on LibriSpeech-960. Hover the bars for the insertion / deletion / substitution breakdown.

RQ1 — Does GRPO beat SFT?

Table I · WER % ↓

On real speech GRPO pays off stacked after SFT (10.27 → 9.49). On synthetic speech the effect is dramatic: GRPO alone cuts WER from 36.71 to 22.09 (−40% rel.), and SFT→GRPO reaches 20.21 (−45% rel.) — recovering most of the synthetic-to-real gap exactly where privacy makes it matter.

RQ2 — How much real data is worth collecting?

Table II · + fixed 54 h synthetic

WER falls steeply over the first hours of real speech, then flattens: ≈⅔ of the reduction comes from the first 5–10 h. Annotate little, gain much.

RQ3 — Do we need all the synthetic speech?

Table III · + fixed 1 h real

Non-monotonic: 25 h is the sweet spot (16.77); the full 54 h drowns the 1 h real anchor (20.32). With 5 h real at 25 h synthetic: 14.56.

RQ4 — Which reward?

Plain 1 − WER is the best WER reward (16.77) — length or CER add-ons mostly add insertions. Optimize exactly the metric you report.

RQ5 — Does selecting synthetic utterances help?

At useful budgets random sampling wins (22.11) — no diversity- or rollout-based selector consistently beats it.

Why does GRPO win?

Behavioral, not representational

Four traces of the same mechanism: GRPO doesn't relearn the audio — it learns when not to continue.

① The hallucination tail disappears

Fig. 1 · insertions vs. reference length (stylized)

Under SFT, even short utterances can trigger tens-to-hundreds of inserted tokens — local defects growing into runaway continuations. GRPO keeps insertions low at every length.

② GRPO learns when to stop

Fig. 2 · token probability around T_last

SFT stays confident after the transcript should end; GRPO's confidence collapses at the boundary. ECE on short utterances: 0.430 → 0.198.

Calibration by utterance length

Table VI · Expected Calibration Error ↓

GRPO lowers ECE in every length bin, most where insertion bursts hurt most — short utterances.

③ Attention stays anchored to the audio

Decoder cross-attention (rows = emitted tokens, columns = audio frames — stylized after Fig. 3). SFT's attention is diffuse and leaks across words, and its inserted tokens ([INS]) attend to nothing in particular. GRPO keeps a sharp, monotonic alignment — and stops. Attention entropy stays lower under GRPO throughout decoding (Fig. 4).

④ …while barely touching the representations

Fig. 5 · layer-wise CKA similarity (stylized)

SFT rewrites early–middle layers (the region tied to the synthetic/real gap); GRPO stays close to the base model, and SFT→GRPO barely moves from SFT: the gain is a policy correction, not new representations.

GRPO = a lightweight behavioral correction

same representations · better stopping calibration · tokens only where audio evidence is stable

Takeaways

What to do when synthetic speech is all you have

✓

Prefer RL over SFT

On synthetic speech, GRPO cuts WER 36.71 → 22.09; SFT→GRPO reaches 20.21 (−45% rel.). On real speech it still adds a small gain (10.27 → 9.49).

✓

Keep the reward simple

r = 1 − WER beats every combination with CER or length terms. Optimize what you evaluate; extras mostly add insertions.

✓

Don't over-engineer the data

Random sampling of ~25 h synthetic wins once the pool is moderate; embedding- and rollout-based selectors don't consistently beat it.

✓

Spend the real budget wisely

5–10 h of real speech capture most of the achievable gain (down to 14.56 with 5 h) — after that, returns diminish fast.

Cite

BibTeX

@misc{kumar2026syntheticspeechhavebetter,
      title={When Synthetic Speech Is All You Have: Better Call GRPO}, 
      author={Shashi Kumar and Yanis Labrak and Hasindri Watawana and Sergio Burdisso and Esaú Villatoro-Tello and Kadri Hacioğlu and Petr Motlicek and Andreas Stolcke},
      year={2026},
      eprint={2607.08409},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2607.08409}, 
}