Set any loudness you want: continuous SPL control hits 83% of the target change in WavLM features
Synopsis
Vocal intensity conversion no longer relies only on a reference utterance or a few discrete categories; it now takes a continuous target sound pressure level in dB SPL. A converter trained as a conditional cycle-consistent GAN works in the feature space of a frozen encoder/decoder pair without parallel data. With WavLM layer 6, it achieves 83% of the target intensity change, and listeners perceive increasing loudness with the target level.
Interpretation
Vocal intensity conversion can now be specified by a continuous physical quantity: the equivalent continuous sound pressure level in dB SPL referred to one meter from the mouth. Earlier systems controlled output either through a reference utterance that already carried the desired effort or through a few discrete categories such as noise levels or effort conditions in their corpora. On the calibrated AVID corpus, the same converter was compared across three frozen encoder/decoder pairs: WavLM layer 6 reached 0.83 of the target intensity change, against 0.62 for log-mel and 0.58 for SpeechTokenizer, with the ranking unchanged relative to each representation's ceiling.
Only a small converter is trained; the encoder and decoder stay frozen, so the output inherits the reconstruction quality of the chosen pair. Related work retrains the representation itself, for example distilling the second layer of SpeechTokenizer from a dedicated vocal-effort encoder. The converter is a six-layer bidirectional Transformer that emits a residual correction per frame, so it preserves alignment and cannot alter duration or speaking rate; training uses 2 s excerpts with target levels drawn uniformly between the 5th and 99th percentiles of the training set.
Training needs neither parallel data nor an intensity predictor inside the generator objective; the target level is imposed by conditioning the discriminator. Using a pretrained intensity predictor as the training signal would set the relation between sequence and level by the predictor rather than by real data. Each discriminator step scores three pairs: a real sequence with its source level, a converted sequence with its target level, and a real sequence with a mismatched level; because levels are continuous, a real sequence counts as valid for every level within a small band of its source level.
A perceptual experiment shows that, on amplitude-normalized stimuli, the perceived loudness of converted speech rises with the target level. Beyond objective measures, it still had to be confirmed that listeners hear a change in vocal intensity rather than in playback amplitude. Eighty-one listeners were recruited and 54 retained; for WavLM layer 6 conversions the perceived level against target level has a slope of 0.69 (95% interval 0.60–0.78), against 0.68 (0.51–0.84) for original recordings, so the two do not differ.
Perspective
What is demonstrated is conversion on the calibrated AVID corpus toward a target equivalent continuous sound pressure level referred to one meter: the recording chain must be calibrated and the mouth-to-microphone distance known, and training needs the source level of every training utterance. Evaluation covers six unseen speakers, 25 sentences, and two repetitions, yielding 300 groups, 1198 source recordings, and 4786 conversions, plus a perceptual experiment. The authors propose estimating training source levels with an automatic intensity predictor so that larger, uncalibrated corpora can be used.
Relative intensity is controlled well, but the absolute perceived level is less consistent: for the same target, it varies between source recordings with a standard deviation of about 6 dB. Conversions displace the fundamental frequency and spectral measures less than original recordings do, which the authors list as room for improvement. Calibrated corpora covering a wide range of vocal intensities are scarce, which limits the training data.
