Skip to main content
Search

# Co-speech Gesture Generation

1 related articles

Subscribe to updates
  1. arXiv

    Speech and gesture leave the two-step pipeline: Gaspard generates both streams in one autoregressive model at a 0.84 real-time factor

    Speech and co-speech gesture synthesis has commonly followed a cascade: generate speech first, then drive gestures from it, so gestures cannot feed back into speech. Gaspard places both modalities in one continuous autoregressive model, predicting the next speech and gesture embeddings at each step with separate flow-matching heads, and uses decoupled classifier-free guidance to regulate each stream's own history and cross-modal conditioning. On Seamless Interaction it yields stronger gesture dynamics and synchrony than a matched cascaded system, with a real-time factor of 0.84 versus 1.04.

You’re all caught up