Speech Synthesis along Perceptual Voice Quality Dimensions
Frederik Rautenberg* , Michael Kuhlmann* ,
Fritz Seebauer† Jana Wiechmann† ,
Petra Wagner† , Reinhold Haeb-Umbach*
* Department of Communications Engineering, Paderborn University
† Phonetics Work Group, Bielefeld University
Abstract:
While expressive speech synthesis or voice conversion systems mainly focus on controlling or manipulating abstract prosodic characteristics of speech, such as emotion or accent, we here address the control of perceptual voice qualities (PVQs) recognized by phonetic experts, which are speech properties at a lower level of abstraction. The ability to manipulate PVQs can be a valuable tool for teaching speech pathologists in training or voice actors. In this paper, we integrate a Conditional Continuous-Normalizing-Flow-based method into a Text-to-Speech system to modify perceptual voice attributes on a continuous scale. Unlike previous approaches, our system avoids direct manipulation of acoustic correlates and instead learns from examples. We demonstrate the system's capability by manipulating four voice qualities: Roughness, breathiness, resonance and weight. Phonetic experts evaluated these modifications, both for seen and unseen speaker conditions. The results highlight both the system's strengths and areas for improvement.
Seen speakers
Example 1
Original Voice
Your browser does not support the audio element.
Breathiness
Low
Your browser does not support the audio element.
Medium
Your browser does not support the audio element.
High
Your browser does not support the audio element.
Roughness
Low
Your browser does not support the audio element.
Medium
Your browser does not support the audio element.
High
Your browser does not support the audio element.
Weight
Low
Your browser does not support the audio element.
Medium
Your browser does not support the audio element.
High
Your browser does not support the audio element.
Resonance
Low
Your browser does not support the audio element.
Medium
Your browser does not support the audio element.
High
Your browser does not support the audio element.
Speaking speed
Low
Your browser does not support the audio element.
Medium
Your browser does not support the audio element.
High
Your browser does not support the audio element.
Example 2
Original Voice
Your browser does not support the audio element.
Breathiness
Low
Your browser does not support the audio element.
Medium
Your browser does not support the audio element.
High
Your browser does not support the audio element.
Roughness
Low
Your browser does not support the audio element.
Medium
Your browser does not support the audio element.
High
Your browser does not support the audio element.
Weight
Low
Your browser does not support the audio element.
Medium
Your browser does not support the audio element.
High
Your browser does not support the audio element.
Resonance
Low
Your browser does not support the audio element.
Medium
Your browser does not support the audio element.
High
Your browser does not support the audio element.
Speaking speed
Low
Your browser does not support the audio element.
Medium
Your browser does not support the audio element.
High
Your browser does not support the audio element.
Unseen speakers
Example 1
Original Voice
Your browser does not support the audio element.
Breathiness
Low
Your browser does not support the audio element.
Medium
Your browser does not support the audio element.
High
Your browser does not support the audio element.
Roughness
Low
Your browser does not support the audio element.
Medium
Your browser does not support the audio element.
High
Your browser does not support the audio element.
Weight
Low
Your browser does not support the audio element.
Medium
Your browser does not support the audio element.
High
Your browser does not support the audio element.
Resonance
Low
Your browser does not support the audio element.
Medium
Your browser does not support the audio element.
High
Your browser does not support the audio element.
Speaking speed
Low
Your browser does not support the audio element.
Medium
Your browser does not support the audio element.
High
Your browser does not support the audio element.
Example 2
Original Voice
Your browser does not support the audio element.
Breathiness
Low
Your browser does not support the audio element.
Medium
Your browser does not support the audio element.
High
Your browser does not support the audio element.
Roughness
Low
Your browser does not support the audio element.
Medium
Your browser does not support the audio element.
High
Your browser does not support the audio element.
Weight
Low
Your browser does not support the audio element.
Medium
Your browser does not support the audio element.
High
Your browser does not support the audio element.
Resonance
Low
Your browser does not support the audio element.
Medium
Your browser does not support the audio element.
High
Your browser does not support the audio element.
Speaking speed
Low
Your browser does not support the audio element.
Medium
Your browser does not support the audio element.
High
Your browser does not support the audio element.