Disentangling Pitch and Creak for Speaker Identity Preservation in Speech Synthesis
Frederik Rautenberg1, Jana Wiechmann2, Petra Wagner2, Reinhold Haeb-Umbach1
1Department of Communications Engineering, Paderborn University
Email: rautenberg@nt.upb.de, haeb@nt.upb.de
2Phonetics Work Group, Bielefeld University
Email: jana.wiechmann@uni-bielefeld.de, petra.wagner@uni-bielefeld.de
Abstract: We introduce a system capable of faithfully modifying the perceptual voice quality of creak while preserving the speaker's perceived identity. While it is well known that high creak probability is typically correlated with low pitch, it is important to note that this is a property observed on a population of speakers but does not necessarily hold across all situations. Disentanglement of pitch from creak is achieved by augmentation of the training dataset of a speech synthesis system with a speaker manipulation block based on conditional continuous normalizing flow. The experiments show greatly improved speaker verification performance over a range of creak manipulation strengths.
Logo

Original Audio

Pitch: 103.19 Hz
Creak: 0.76
Factorbase-flowadapted-flowcombined-flow
-1.00
Pitch: 183.19 Hz
Creak: 0.317
Pitch: 125.57 Hz
Creak: 0.506
Pitch: 139.42 Hz
Creak: 0.427
-0.75
Pitch: 171.85 Hz
Creak: 0.407
Pitch: 121.14 Hz
Creak: 0.544
Pitch: 129.17 Hz
Creak: 0.520
-0.50
Pitch: 152.86 Hz
Creak: 0.529
Pitch: 114.55 Hz
Creak: 0.643
Pitch: 120.67 Hz
Creak: 0.612
-0.25
Pitch: 128.93 Hz
Creak: 0.635
Pitch: 109.18 Hz
Creak: 0.710
Pitch: 112.17 Hz
Creak: 0.709
0.00
Pitch: 103.69 Hz
Creak: 0.756
Pitch: 103.69 Hz
Creak: 0.755
Pitch: 103.69 Hz
Creak: 0.755
0.25
Pitch: 84.78 Hz
Creak: 0.804
Pitch: 96.41 Hz
Creak: 0.779
Pitch: 94.29 Hz
Creak: 0.802
0.50
Pitch: 76.43 Hz
Creak: 0.769
Pitch: 92.46 Hz
Creak: 0.803
Pitch: 89.67 Hz
Creak: 0.826
0.75
Pitch: 73.09 Hz
Creak: 0.764
Pitch: 95.13 Hz
Creak: 0.799
Pitch: 84.81 Hz
Creak: 0.836
1.00
Pitch: 69.23 Hz
Creak: 0.776
Pitch: 102.00 Hz
Creak: 0.799
Pitch: 88.09 Hz
Creak: 0.846

Original Audio

Pitch: 133.00 Hz
Creak: 0.720
Factorbase-flowadapted-flowcombined-flow
-1.00
Pitch: 172.86 Hz
Creak: 0.337
Pitch: 137.06 Hz
Creak: 0.435
Pitch: 146.46 Hz
Creak: 0.392
-0.75
Pitch: 158.88 Hz
Creak: 0.427
Pitch: 131.16 Hz
Creak: 0.480
Pitch: 136.79 Hz
Creak: 0.450
-0.50
Pitch: 143.86 Hz
Creak: 0.498
Pitch: 125.68 Hz
Creak: 0.541
Pitch: 129.62 Hz
Creak: 0.511
-0.25
Pitch: 129.13 Hz
Creak: 0.571
Pitch: 120.49 Hz
Creak: 0.592
Pitch: 121.93 Hz
Creak: 0.591
0.00
Pitch: 114.18 Hz
Creak: 0.675
Pitch: 114.18 Hz
Creak: 0.675
Pitch: 114.18 Hz
Creak: 0.674
0.25
Pitch: 101.92 Hz
Creak: 0.761
Pitch: 108.21 Hz
Creak: 0.755
Pitch: 108.52 Hz
Creak: 0.753
0.50
Pitch: 92.60 Hz
Creak: 0.786
Pitch: 103.15 Hz
Creak: 0.788
Pitch: 100.37 Hz
Creak: 0.802
0.75
Pitch: 83.58 Hz
Creak: 0.807
Pitch: 97.08 Hz
Creak: 0.809
Pitch: 92.68 Hz
Creak: 0.818
1.00
Pitch: 76.04 Hz
Creak: 0.816
Pitch: 91.76 Hz
Creak: 0.824
Pitch: 86.38 Hz
Creak: 0.826

Original Audio

Pitch: 199.00 Hz
Creak: 0.41
Factorbase-flowadapted-flowcombined-flow
-1.00
Pitch: 201.56 Hz
Creak: 0.240
Pitch: 180.88 Hz
Creak: 0.301
Pitch: 186.94 Hz
Creak: 0.257
-0.75
Pitch: 200.03 Hz
Creak: 0.257
Pitch: 187.76 Hz
Creak: 0.297
Pitch: 190.51 Hz
Creak: 0.268
-0.50
Pitch: 197.92 Hz
Creak: 0.280
Pitch: 189.80 Hz
Creak: 0.303
Pitch: 192.68 Hz
Creak: 0.284
-0.25
Pitch: 193.50 Hz
Creak: 0.304
Pitch: 187.21 Hz
Creak: 0.322
Pitch: 189.62 Hz
Creak: 0.316
0.00
Pitch: 190.51 Hz
Creak: 0.350
Pitch: 190.52 Hz
Creak: 0.350
Pitch: 190.52 Hz
Creak: 0.349
0.25
Pitch: 181.67 Hz
Creak: 0.410
Pitch: 188.85 Hz
Creak: 0.391
Pitch: 185.34 Hz
Creak: 0.411
0.50
Pitch: 158.71 Hz
Creak: 0.537
Pitch: 186.68 Hz
Creak: 0.451
Pitch: 177.34 Hz
Creak: 0.515
0.75
Pitch: 134.69 Hz
Creak: 0.685
Pitch: 184.41 Hz
Creak: 0.526
Pitch: 163.59 Hz
Creak: 0.630
1.00
Pitch: 103.13 Hz
Creak: 0.790
Pitch: 187.72 Hz
Creak: 0.612
Pitch: 163.81 Hz
Creak: 0.724

Original Audio

Pitch: 188.00 Hz
Creak: 0.38
Factor base-flow adapted-flow combined-flow
-1.00
Pitch: 200.70 Hz
Creak: 0.210
Pitch: 183.93 Hz
Creak: 0.262
Pitch: 186.02 Hz
Creak: 0.230
-0.75
Pitch: 199.16 Hz
Creak: 0.235
Pitch: 186.22 Hz
Creak: 0.264
Pitch: 187.78 Hz
Creak: 0.249
-0.50
Pitch: 196.82 Hz
Creak: 0.246
Pitch: 187.04 Hz
Creak: 0.278
Pitch: 188.38 Hz
Creak: 0.254
-0.25
Pitch: 191.93 Hz
Creak: 0.276
Pitch: 186.01 Hz
Creak: 0.290
Pitch: 187.06 Hz
Creak: 0.287
0.00
Pitch: 183.94 Hz
Creak: 0.327
Pitch: 183.95 Hz
Creak: 0.327
Pitch: 183.95 Hz
Creak: 0.327
0.25
Pitch: 167.17 Hz
Creak: 0.406
Pitch: 180.86 Hz
Creak: 0.359
Pitch: 178.56 Hz
Creak: 0.380
0.50
Pitch: 134.28 Hz
Creak: 0.582
Pitch: 175.61 Hz
Creak: 0.406
Pitch: 172.89 Hz
Creak: 0.445
0.75
Pitch: 99.37 Hz
Creak: 0.712
Pitch: 172.91 Hz
Creak: 0.484
Pitch: 167.93 Hz
Creak: 0.518
1.00
Pitch: 84.75 Hz
Creak: 0.742
Pitch: 166.63 Hz
Creak: 0.558
Pitch: 149.76 Hz
Creak: 0.656