- 著者
- Yuki SAITO Kei AKUZAWA Kentaro TACHIBANA
- 出版者
- The Institute of Electronics, Information and Communication Engineers
- 雑誌
- IEICE TRANSACTIONS on Information and Systems (ISSN:09168532)
- 巻号頁・発行日
- vol.E103-D, no.9, pp.1978-1987, 2020-09-01
This paper presents a method for many-to-one voice conversion using phonetic posteriorgrams (PPGs) based on an adversarial training of deep neural networks (DNNs). A conventional method for many-to-one VC can learn a mapping function from input acoustic features to target acoustic features through separately trained DNN-based speech recognition and synthesis models. However, 1) the differences among speakers observed in PPGs and 2) an over-smoothing effect of generated acoustic features degrade the converted speech quality. Our method performs a domain-adversarial training of the recognition model for reducing the PPG differences. In addition, it incorporates a generative adversarial network into the training of the synthesis model for alleviating the over-smoothing effect. Unlike the conventional method, ours jointly trains the recognition and synthesis models so that they are optimized for many-to-one VC. Experimental evaluation demonstrates that the proposed method significantly improves the converted speech quality compared with conventional VC methods.
- 著者
- Yuki SAITO Kei AKUZAWA Kentaro TACHIBANA
- 出版者
- The Institute of Electronics, Information and Communication Engineers
- 雑誌
- IEICE Transactions on Information and Systems (ISSN:09168532)
- 巻号頁・発行日
- vol.E103.D, no.9, pp.1978-1987, 2020-09-01 (Released:2020-09-01)
- 参考文献数
- 53
This paper presents a method for many-to-one voice conversion using phonetic posteriorgrams (PPGs) based on an adversarial training of deep neural networks (DNNs). A conventional method for many-to-one VC can learn a mapping function from input acoustic features to target acoustic features through separately trained DNN-based speech recognition and synthesis models. However, 1) the differences among speakers observed in PPGs and 2) an over-smoothing effect of generated acoustic features degrade the converted speech quality. Our method performs a domain-adversarial training of the recognition model for reducing the PPG differences. In addition, it incorporates a generative adversarial network into the training of the synthesis model for alleviating the over-smoothing effect. Unlike the conventional method, ours jointly trains the recognition and synthesis models so that they are optimized for many-to-one VC. Experimental evaluation demonstrates that the proposed method significantly improves the converted speech quality compared with conventional VC methods.