Skip to main content
Search

# Voice Conversion

2 related articles

Subscribe to updates
  1. arXiv

    Dubbing plus voice conversion yields four-way aligned speech pairs, lifting expressive translation in English–German and English–Spanish

    Expressive speech-to-speech translation needs training pairs aligned in content, prosody, timing and speaker identity at once, yet existing resources leave at least one dimension unmatched. This work combines professional dubbing, which supplies human-performed correspondence, with multilingual zero-shot voice conversion to build four-way aligned English–German and English–Spanish pairs, and tests that supervision on TransVIP, Dub-S2ST and an end-to-end model called DirectS2ST. On the independent mExpresso evaluation set, prosody and speaker similarity rise for all three model families, DirectS2ST reaches the best prosody and pause scores in both directions, and nine listeners raise its expressiveness rating by 1.29.

  2. arXiv

    21Codec moves speaker identity to the decoder: streaming voices stop drifting chunk to chunk, and can change mid-stream

    Neural codecs for streaming text-to-speech usually infer speaker identity from the discrete token stream, and chunked decoding re-infers it from each local chunk alone, so the voice drifts within one utterance. 21Codec suppresses speaker information in the tokens and instead conditions every frame on an external speaker embedding at the decoder, which holds identity during streaming and allows a speaker change mid-stream. On unseen speakers, re-voiced output sits closer to the reference speaker than to the source; the cost is a higher word error rate than the unconditional version.

You’re all caught up