文献一覧: Shinnosuke Takamichi (著者)

15 0 0 0 OA Real-Time Full-Band Voice Conversion with Sub-Band Modeling and Data-Driven Phase Estimation of Spectral Differentials

著者: Takaaki SAEKI Yuki SAITO Shinnosuke TAKAMICHI Hiroshi SARUWATARI
出版者: The Institute of Electronics, Information and Communication Engineers
雑誌: IEICE Transactions on Information and Systems (ISSN:09168532)
巻号頁・発行日: vol.E104.D, no.7, pp.1002-1016, 2021-07-01 (Released:2021-07-01)
参考文献数: 41
被引用文献数: 2

This paper proposes two high-fidelity and computationally efficient neural voice conversion (VC) methods based on a direct waveform modification using spectral differentials. The conventional spectral-differential VC method with a minimum-phase filter achieves high-quality conversion for narrow-band (16 kHz-sampled) VC but requires heavy computational cost in filtering. This is because the minimum phase obtained using a fixed lifter of the Hilbert transform often results in a long-tap filter. Furthermore, when we extend the method to full-band (48 kHz-sampled) VC, the computational cost is heavy due to increased sampling points, and the converted-speech quality degrades due to large fluctuations in the high-frequency band. To construct a short-tap filter, we propose a lifter-training method for data-driven phase reconstruction that trains a lifter of the Hilbert transform by taking into account filter truncation. We also propose a frequency-band-wise modeling method based on sub-band multi-rate signal processing (sub-band modeling method) for full-band VC. It enhances the computational efficiency by reducing sampling points of signals converted with filtering and improves converted-speech quality by modeling only the low-frequency band. We conducted several objective and subjective evaluations to investigate the effectiveness of the proposed methods through implementation of the real-time, online, full-band VC system we developed, which is based on the proposed methods. The results indicate that 1) the proposed lifter-training method for narrow-band VC can shorten the tap length to 1/16 without degrading the converted-speech quality, and 2) the proposed sub-band modeling method for full-band VC can improve the converted-speech quality while reducing the computational cost, and 3) our real-time, online, full-band VC system can convert 48 kHz-sampled speech in real time attaining the converted speech with a 3.6 out of 5.0 mean opinion score of naturalness.

2023-11-11 13:06:45
15 + 39 Twitter

13 0 0 0 OA JSUT and JVS: Free Japanese voice corpora for accelerating speech synthesis research

著者: Shinnosuke Takamichi Ryosuke Sonobe Kentaro Mitsui Yuki Saito Tomoki Koriyama Naoko Tanji Hiroshi Saruwatari
出版者: ACOUSTICAL SOCIETY OF JAPAN
雑誌: Acoustical Science and Technology (ISSN:13463969)
巻号頁・発行日: vol.41, no.5, pp.761-768, 2020-09-01 (Released:2020-09-01)
参考文献数: 50
被引用文献数: 17

In this paper, we develop two corpora for speech synthesis research. Thanks to improvements in machine learning techniques, including deep learning, speech synthesis is becoming a machine learning task. To accelerate speech synthesis research, we aim at developing Japanese voice corpora reasonably accessible from not only academic institutions but also commercial companies. In this paper, we construct the JSUT and JVS corpora. They are designed mainly for text-to-speech synthesis and voice conversion, respectively. The JSUT corpus contains 10 hours of reading-style speech uttered by a single speaker, and the JVS corpus contains 30 hours containing three styles of speech uttered by 100 speakers. This paper describes how we designed the corpora and summarizes the specifications. The corpora are available at our project pages.

2021-11-27 12:53:03
13 + 27 Twitter

12 0 0 0 OA Perception analysis of inter-singer similarity in Japanese song

著者: Hiroki Tamaru Shinnosuke Takamichi Hiroshi Saruwatari
出版者: ACOUSTICAL SOCIETY OF JAPAN
雑誌: Acoustical Science and Technology (ISSN:13463969)
巻号頁・発行日: vol.41, no.5, pp.804-807, 2020-09-01 (Released:2020-09-01)
参考文献数: 16
被引用文献数: 1

2020-09-02 03:09:17
12 + 18 Twitter

4 0 0 0 OA Generative Moment Matching Network-Based Neural Double-Tracking for Synthesized and Natural Singing Voices

著者: Hiroki TAMARU Yuki SAITO Shinnosuke TAKAMICHI Tomoki KORIYAMA Hiroshi SARUWATARI
出版者: The Institute of Electronics, Information and Communication Engineers
雑誌: IEICE Transactions on Information and Systems (ISSN:09168532)
巻号頁・発行日: vol.E103.D, no.3, pp.639-647, 2020-03-01 (Released:2020-03-01)
参考文献数: 32
被引用文献数: 3

This paper proposes a generative moment matching network (GMMN)-based post-filtering method for providing inter-utterance pitch variation to singing voices and discusses its application to our developed mixing method called neural double-tracking (NDT). When a human singer sings and records the same song twice, there is a difference between the two recordings. The difference, which is called inter-utterance variation, enriches the performer's musical expression and the audience's experience. For example, it makes every concert special because it never recurs in exactly the same manner. Inter-utterance variation enables a mixing method called double-tracking (DT). With DT, the same phrase is recorded twice, then the two recordings are mixed to give richness to singing voices. However, in synthesized singing voices, which are commonly used to create music, there is no inter-utterance variation because the synthesis process is deterministic. There is also no inter-utterance variation when only one voice is recorded. Although there is a signal processing-based method called artificial DT (ADT) to layer singing voices, the signal processing results in unnatural sound artifacts. To solve these problems, we propose a post-filtering method for randomly modulating synthesized or natural singing voices as if the singer sang again. The post-filter built with our method models the inter-utterance pitch variation of human singing voices using a conditional GMMN. Evaluation results indicate that 1) the proposed method provides perceptible and natural inter-utterance variation to synthesized singing voices and that 2) our NDT exhibits higher double-trackedness than ADT when applied to both synthesized and natural singing voices.

2020-10-29 11:14:58
4 + 5 Twitter

3 0 0 0 OA Perceived azimuth-based creditability and self-reported confidence for sound localization experiments using crowdsourcing

著者: Shinnosuke Takamichi Daisuke Morikawa
出版者: ACOUSTICAL SOCIETY OF JAPAN
雑誌: Acoustical Science and Technology (ISSN:13463969)
巻号頁・発行日: vol.40, no.2, pp.142-143, 2019-03-01 (Released:2019-03-01)
参考文献数: 3
被引用文献数: 1

2021-04-05 23:04:21
3 + 1 Twitter

3 0 0 0 DNN-Based Full-Band Speech Synthesis Using GMM Approximation of Spectral Envelope

著者: Junya KOGUCHI Shinnosuke TAKAMICHI Masanori MORISE Hiroshi SARUWATARI Shigeki SAGAYAMA
出版者: The Institute of Electronics, Information and Communication Engineers
雑誌: IEICE TRANSACTIONS on Information and Systems (ISSN:09168532)
巻号頁・発行日: vol.E103-D, no.12, pp.2673-2681, 2020-12-01
被引用文献数: 2

We propose a speech analysis-synthesis and deep neural network (DNN)-based text-to-speech (TTS) synthesis framework using Gaussian mixture model (GMM)-based approximation of full-band spectral envelopes. GMMs have excellent properties as acoustic features in statistic parametric speech synthesis. Each Gaussian function of a GMM fits the local resonance of the spectrum. The GMM retains the fine spectral envelope and achieve high controllability of the structure. However, since conventional speech analysis methods (i.e., GMM parameter estimation) have been formulated for a narrow-band speech, they degrade the quality of synthetic speech. Moreover, a DNN-based TTS synthesis method using GMM-based approximation has not been formulated in spite of its excellent expressive ability. Therefore, we employ peak-picking-based initialization for full-band speech analysis to provide better initialization for iterative estimation of the GMM parameters. We introduce not only prediction error of GMM parameters but also reconstruction error of the spectral envelopes as objective criteria for training DNN. Furthermore, we propose a method for multi-task learning based on minimizing these errors simultaneously. We also propose a post-filter based on variance scaling of the GMM for our framework to enhance synthetic speech. Experimental results from evaluating our framework indicated that 1) the initialization method of our framework outperformed the conventional one in the quality of analysis-synthesized speech; 2) introducing the reconstruction error in DNN training significantly improved the synthetic speech; 3) our variance-scaling-based post-filter further improved the synthetic speech.

2020-12-01 12:27:48
3 + 12 Twitter

3 0 0 0 OA Voice Conversion Using Input-to-Output Highway Networks

著者: Yuki SAITO Shinnosuke TAKAMICHI Hiroshi SARUWATARI
出版者: 一般社団法人電子情報通信学会
雑誌: IEICE Transactions on Information and Systems (ISSN:09168532)
巻号頁・発行日: vol.E100.D, no.8, pp.1925-1928, 2017-08-01 (Released:2017-08-01)
参考文献数: 20
被引用文献数: 19

This paper proposes Deep Neural Network (DNN)-based Voice Conversion (VC) using input-to-output highway networks. VC is a speech synthesis technique that converts input features into output speech parameters, and DNN-based acoustic models for VC are used to estimate the output speech parameters from the input speech parameters. Given that the input and output are often in the same domain (e.g., cepstrum) in VC, this paper proposes a VC using highway networks connected from the input to output. The acoustic models predict the weighted spectral differentials between the input and output spectral parameters. The architecture not only alleviates over-smoothing effects that degrade speech quality, but also effectively represents the characteristics of spectral parameters. The experimental results demonstrate that the proposed architecture outperforms Feed-Forward neural networks in terms of the speech quality and speaker individuality of the converted speech.

2017-08-01 12:37:03
3 + 2 Twitter

2 0 0 0 OA Non-parallel and many-to-many voice conversion using variational autoencoders integrating speech recognition and speaker verification

著者: Yuki Saito Taiki Nakamura Yusuke Ijima Kyosuke Nishida Shinnosuke Takamichi
出版者: ACOUSTICAL SOCIETY OF JAPAN
雑誌: Acoustical Science and Technology (ISSN:13463969)
巻号頁・発行日: vol.42, no.1, pp.1-11, 2021-01-01 (Released:2021-01-01)
参考文献数: 34
被引用文献数: 1

We propose non-parallel and many-to-many voice conversion (VC) using variational autoencoders (VAEs) that constructs VC models for converting arbitrary speakers' characteristics into those of other arbitrary speakers without parallel speech corpora for training the models. Although VAEs conditioned by one-hot coded speaker codes can achieve non-parallel VC, the phonetic contents of the converted speech tend to vanish, resulting in degraded speech quality. Another issue is that they cannot deal with unseen speakers not included in training corpora. To overcome these issues, we incorporate deep-neural-network-based automatic speech recognition (ASR) and automatic speaker verification (ASV) into the VAE-based VC. Since phonetic contents are given as phonetic posteriorgrams predicted from the ASR models, the proposed VC can overcome the quality degradation. Our VC utilizes d-vectors extracted from the ASV models as continuous speaker representations that can deal with unseen speakers. Experimental results demonstrate that our VC outperforms the conventional VAE-based VC in terms of mel-cepstral distortion and converted speech quality. We also investigate the effects of hyperparameters in our VC and reveal that 1) a large d-vector dimensionality that gives the better ASV performance does not necessarily improve converted speech quality, and 2) a large number of pre-stored speakers improves the quality.

2021-01-01 10:20:32
2 + 4 Twitter

1 0 0 0 DNN-Based Low-Musical-Noise Single-Channel Speech Enhancement Based on Higher-Order-Moments Matching

著者: Satoshi MIZOGUCHI Yuki SAITO Shinnosuke TAKAMICHI Hiroshi SARUWATARI
出版者: The Institute of Electronics, Information and Communication Engineers
雑誌: IEICE TRANSACTIONS on Information and Systems (ISSN:09168532)
巻号頁・発行日: vol.E104-D, no.11, pp.1971-1980, 2021-11-01
被引用文献数: 1

We propose deep neural network (DNN)-based speech enhancement that reduces musical noise and achieves better auditory impressions. The musical noise is an artifact generated by nonlinear signal processing and negatively affects the auditory impressions. We aim to develop musical-noise-free speech enhancement methods that suppress the musical noise generation and produce perceptually-comfortable enhanced speech. DNN-based speech enhancement using a soft mask achieves high noise reduction but generates musical noise in non-speech regions. Therefore, first, we define kurtosis matching for DNN-based low-musical-noise speech enhancement. Kurtosis is the fourth-order moment and is known to correlate with the amount of musical noise. The kurtosis matching is a penalty term of the DNN training and works to reduce the amount of musical noise. We further extend this scheme to standardized-moment matching. The extended scheme involves using moments whose orders are higher than kurtosis and generalizes the conventional musical-noise-free method based on kurtosis matching. We formulate standardized-moment matching and explore how effectively the higher-order moments reduce the amount of musical noise. Experimental evaluation results 1) demonstrate that kurtosis matching can reduce musical noise without negatively affecting noise suppression and 2) newly reveal that the sixth-moment matching also achieves low-musical-noise speech enhancement as well as kurtosis matching.

2021-11-01 12:47:10
1 + 3 Twitter

1 0 0 0 Real-Time Full-Band Voice Conversion with Sub-Band Modeling and Data-Driven Phase Estimation of Spectral Differentials

著者: Takaaki SAEKI Yuki SAITO Shinnosuke TAKAMICHI Hiroshi SARUWATARI
出版者: The Institute of Electronics, Information and Communication Engineers
雑誌: IEICE TRANSACTIONS on Information and Systems (ISSN:09168532)
巻号頁・発行日: vol.E104-D, no.7, pp.1002-1016, 2021-07-01
被引用文献数: 2

2021-07-01 08:08:58
1 + 4 Twitter

1 0 0 0 Voice Conversion Using Input-to-Output Highway Networks

著者: Yuki SAITO Shinnosuke TAKAMICHI Hiroshi SARUWATARI
出版者: The Institute of Electronics, Information and Communication Engineers
雑誌: IEICE TRANSACTIONS on Information and Systems (ISSN:09168532)
巻号頁・発行日: vol.E100-D, no.8, pp.1925-1928, 2017-08-01

2017-08-01 00:04:55
1 + 0 Twitter

http://search.ieice.org/bin/summary.php?id=e100-d_8_1925&category=D&year=2017&lang=E&abst=