文献一覧: Hiroshi Saruwatari (著者)

15 0 0 0 OA Real-Time Full-Band Voice Conversion with Sub-Band Modeling and Data-Driven Phase Estimation of Spectral Differentials

著者: Takaaki SAEKI Yuki SAITO Shinnosuke TAKAMICHI Hiroshi SARUWATARI
出版者: The Institute of Electronics, Information and Communication Engineers
雑誌: IEICE Transactions on Information and Systems (ISSN:09168532)
巻号頁・発行日: vol.E104.D, no.7, pp.1002-1016, 2021-07-01 (Released:2021-07-01)
参考文献数: 41
被引用文献数: 2

This paper proposes two high-fidelity and computationally efficient neural voice conversion (VC) methods based on a direct waveform modification using spectral differentials. The conventional spectral-differential VC method with a minimum-phase filter achieves high-quality conversion for narrow-band (16 kHz-sampled) VC but requires heavy computational cost in filtering. This is because the minimum phase obtained using a fixed lifter of the Hilbert transform often results in a long-tap filter. Furthermore, when we extend the method to full-band (48 kHz-sampled) VC, the computational cost is heavy due to increased sampling points, and the converted-speech quality degrades due to large fluctuations in the high-frequency band. To construct a short-tap filter, we propose a lifter-training method for data-driven phase reconstruction that trains a lifter of the Hilbert transform by taking into account filter truncation. We also propose a frequency-band-wise modeling method based on sub-band multi-rate signal processing (sub-band modeling method) for full-band VC. It enhances the computational efficiency by reducing sampling points of signals converted with filtering and improves converted-speech quality by modeling only the low-frequency band. We conducted several objective and subjective evaluations to investigate the effectiveness of the proposed methods through implementation of the real-time, online, full-band VC system we developed, which is based on the proposed methods. The results indicate that 1) the proposed lifter-training method for narrow-band VC can shorten the tap length to 1/16 without degrading the converted-speech quality, and 2) the proposed sub-band modeling method for full-band VC can improve the converted-speech quality while reducing the computational cost, and 3) our real-time, online, full-band VC system can convert 48 kHz-sampled speech in real time attaining the converted speech with a 3.6 out of 5.0 mean opinion score of naturalness.

2023-11-11 13:06:45
15 + 39 Twitter

13 0 0 0 OA JSUT and JVS: Free Japanese voice corpora for accelerating speech synthesis research

著者: Shinnosuke Takamichi Ryosuke Sonobe Kentaro Mitsui Yuki Saito Tomoki Koriyama Naoko Tanji Hiroshi Saruwatari
出版者: ACOUSTICAL SOCIETY OF JAPAN
雑誌: Acoustical Science and Technology (ISSN:13463969)
巻号頁・発行日: vol.41, no.5, pp.761-768, 2020-09-01 (Released:2020-09-01)
参考文献数: 50
被引用文献数: 17

In this paper, we develop two corpora for speech synthesis research. Thanks to improvements in machine learning techniques, including deep learning, speech synthesis is becoming a machine learning task. To accelerate speech synthesis research, we aim at developing Japanese voice corpora reasonably accessible from not only academic institutions but also commercial companies. In this paper, we construct the JSUT and JVS corpora. They are designed mainly for text-to-speech synthesis and voice conversion, respectively. The JSUT corpus contains 10 hours of reading-style speech uttered by a single speaker, and the JVS corpus contains 30 hours containing three styles of speech uttered by 100 speakers. This paper describes how we designed the corpora and summarizes the specifications. The corpora are available at our project pages.

2021-11-27 12:53:03
13 + 27 Twitter

12 0 0 0 OA Perception analysis of inter-singer similarity in Japanese song

著者: Hiroki Tamaru Shinnosuke Takamichi Hiroshi Saruwatari
出版者: ACOUSTICAL SOCIETY OF JAPAN
雑誌: Acoustical Science and Technology (ISSN:13463969)
巻号頁・発行日: vol.41, no.5, pp.804-807, 2020-09-01 (Released:2020-09-01)
参考文献数: 16
被引用文献数: 1

2020-09-02 03:09:17
12 + 18 Twitter

4 0 0 0 OA Generative Moment Matching Network-Based Neural Double-Tracking for Synthesized and Natural Singing Voices

著者: Hiroki TAMARU Yuki SAITO Shinnosuke TAKAMICHI Tomoki KORIYAMA Hiroshi SARUWATARI
出版者: The Institute of Electronics, Information and Communication Engineers
雑誌: IEICE Transactions on Information and Systems (ISSN:09168532)
巻号頁・発行日: vol.E103.D, no.3, pp.639-647, 2020-03-01 (Released:2020-03-01)
参考文献数: 32
被引用文献数: 3

This paper proposes a generative moment matching network (GMMN)-based post-filtering method for providing inter-utterance pitch variation to singing voices and discusses its application to our developed mixing method called neural double-tracking (NDT). When a human singer sings and records the same song twice, there is a difference between the two recordings. The difference, which is called inter-utterance variation, enriches the performer's musical expression and the audience's experience. For example, it makes every concert special because it never recurs in exactly the same manner. Inter-utterance variation enables a mixing method called double-tracking (DT). With DT, the same phrase is recorded twice, then the two recordings are mixed to give richness to singing voices. However, in synthesized singing voices, which are commonly used to create music, there is no inter-utterance variation because the synthesis process is deterministic. There is also no inter-utterance variation when only one voice is recorded. Although there is a signal processing-based method called artificial DT (ADT) to layer singing voices, the signal processing results in unnatural sound artifacts. To solve these problems, we propose a post-filtering method for randomly modulating synthesized or natural singing voices as if the singer sang again. The post-filter built with our method models the inter-utterance pitch variation of human singing voices using a conditional GMMN. Evaluation results indicate that 1) the proposed method provides perceptible and natural inter-utterance variation to synthesized singing voices and that 2) our NDT exhibits higher double-trackedness than ADT when applied to both synthesized and natural singing voices.

2020-10-29 11:14:58
4 + 5 Twitter

4 0 0 0 OA Independent Low-Rank Matrix Analysis Based on Generalized Kullback-Leibler Divergence

著者: Shinichi MOGAMI Yoshiki MITSUI Norihiro TAKAMUNE Daichi KITAMURA Hiroshi SARUWATARI Yu TAKAHASHI Kazunobu KONDO Hiroaki NAKAJIMA Hirokazu KAMEOKA
出版者: The Institute of Electronics, Information and Communication Engineers
雑誌: IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences (ISSN:09168508)
巻号頁・発行日: vol.E102-A, no.2, pp.458-463, 2019-02-01
被引用文献数: 5

In this letter, we propose a new blind source separation method, independent low-rank matrix analysis based on generalized Kullback-Leibler divergence. This method assumes a time-frequency-varying complex Poisson distribution as the source generative model, which yields convex optimization in the spectrogram estimation. The experimental evaluation confirms the proposed method's efficacy.

2019-02-01 19:09:01
4 + 7 Twitter

3 0 0 0 Convex and Differentiable Formulation for Inverse Problems in Hilbert Spaces with Nonlinear Clipping Effects

著者: Natsuki UENO Shoichi KOYAMA Hiroshi SARUWATARI
出版者: The Institute of Electronics, Information and Communication Engineers
雑誌: IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences (ISSN:09168508)
巻号頁・発行日: pp.2021EAP1004, (Released:2021-02-25)

We propose a useful formulation for ill-posed inverse problems in Hilbert spaces with nonlinear clipping effects. Ill-posed inverse problems are often formulated as optimization problems, and nonlinear clipping effects may cause nonconvexity or nondifferentiability of the objective functions in the case of commonly used regularized least squares. To overcome these difficulties, we present a tractable formulation in which the objective function is convex and differentiable with respect to optimization variables, on the basis of the Bregman divergence associated with the primitive function of the clipping function. By using this formulation in combination with the representer theorem, we need only to deal with a finite-dimensional, convex, and differentiable optimization problem, which can be solved by well-established algorithms. We also show two practical examples of inverse problems where our theory can be applied, estimation of band-limited signals and time-harmonic acoustic fields, and evaluate the validity of our theory by numerical simulations.

2021-03-01 11:43:47
3 + 10 Twitter

3 0 0 0 DNN-Based Full-Band Speech Synthesis Using GMM Approximation of Spectral Envelope

著者: Junya KOGUCHI Shinnosuke TAKAMICHI Masanori MORISE Hiroshi SARUWATARI Shigeki SAGAYAMA
出版者: The Institute of Electronics, Information and Communication Engineers
雑誌: IEICE TRANSACTIONS on Information and Systems (ISSN:09168532)
巻号頁・発行日: vol.E103-D, no.12, pp.2673-2681, 2020-12-01
被引用文献数: 2

We propose a speech analysis-synthesis and deep neural network (DNN)-based text-to-speech (TTS) synthesis framework using Gaussian mixture model (GMM)-based approximation of full-band spectral envelopes. GMMs have excellent properties as acoustic features in statistic parametric speech synthesis. Each Gaussian function of a GMM fits the local resonance of the spectrum. The GMM retains the fine spectral envelope and achieve high controllability of the structure. However, since conventional speech analysis methods (i.e., GMM parameter estimation) have been formulated for a narrow-band speech, they degrade the quality of synthetic speech. Moreover, a DNN-based TTS synthesis method using GMM-based approximation has not been formulated in spite of its excellent expressive ability. Therefore, we employ peak-picking-based initialization for full-band speech analysis to provide better initialization for iterative estimation of the GMM parameters. We introduce not only prediction error of GMM parameters but also reconstruction error of the spectral envelopes as objective criteria for training DNN. Furthermore, we propose a method for multi-task learning based on minimizing these errors simultaneously. We also propose a post-filter based on variance scaling of the GMM for our framework to enhance synthetic speech. Experimental results from evaluating our framework indicated that 1) the initialization method of our framework outperformed the conventional one in the quality of analysis-synthesized speech; 2) introducing the reconstruction error in DNN training significantly improved the synthetic speech; 3) our variance-scaling-based post-filter further improved the synthetic speech.

2020-12-01 12:27:48
3 + 12 Twitter

3 0 0 0 OA Voice Conversion Using Input-to-Output Highway Networks

著者: Yuki SAITO Shinnosuke TAKAMICHI Hiroshi SARUWATARI
出版者: 一般社団法人電子情報通信学会
雑誌: IEICE Transactions on Information and Systems (ISSN:09168532)
巻号頁・発行日: vol.E100.D, no.8, pp.1925-1928, 2017-08-01 (Released:2017-08-01)
参考文献数: 20
被引用文献数: 19

This paper proposes Deep Neural Network (DNN)-based Voice Conversion (VC) using input-to-output highway networks. VC is a speech synthesis technique that converts input features into output speech parameters, and DNN-based acoustic models for VC are used to estimate the output speech parameters from the input speech parameters. Given that the input and output are often in the same domain (e.g., cepstrum) in VC, this paper proposes a VC using highway networks connected from the input to output. The acoustic models predict the weighted spectral differentials between the input and output spectral parameters. The architecture not only alleviates over-smoothing effects that degrade speech quality, but also effectively represents the characteristics of spectral parameters. The experimental results demonstrate that the proposed architecture outperforms Feed-Forward neural networks in terms of the speech quality and speaker individuality of the converted speech.

2017-08-01 12:37:03
3 + 2 Twitter

2 0 0 0 OA Convex and Differentiable Formulation for Inverse Problems in Hilbert Spaces with Nonlinear Clipping Effects

著者: Natsuki UENO Shoichi KOYAMA Hiroshi SARUWATARI
出版者: The Institute of Electronics, Information and Communication Engineers
雑誌: IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences (ISSN:09168508)
巻号頁・発行日: vol.E104-A, no.9, pp.1293-1303, 2021-09-01

2021-09-06 12:40:38
2 + 3 Twitter

1 0 0 0 DNN-Based Low-Musical-Noise Single-Channel Speech Enhancement Based on Higher-Order-Moments Matching

著者: Satoshi MIZOGUCHI Yuki SAITO Shinnosuke TAKAMICHI Hiroshi SARUWATARI
出版者: The Institute of Electronics, Information and Communication Engineers
雑誌: IEICE TRANSACTIONS on Information and Systems (ISSN:09168532)
巻号頁・発行日: vol.E104-D, no.11, pp.1971-1980, 2021-11-01
被引用文献数: 1

We propose deep neural network (DNN)-based speech enhancement that reduces musical noise and achieves better auditory impressions. The musical noise is an artifact generated by nonlinear signal processing and negatively affects the auditory impressions. We aim to develop musical-noise-free speech enhancement methods that suppress the musical noise generation and produce perceptually-comfortable enhanced speech. DNN-based speech enhancement using a soft mask achieves high noise reduction but generates musical noise in non-speech regions. Therefore, first, we define kurtosis matching for DNN-based low-musical-noise speech enhancement. Kurtosis is the fourth-order moment and is known to correlate with the amount of musical noise. The kurtosis matching is a penalty term of the DNN training and works to reduce the amount of musical noise. We further extend this scheme to standardized-moment matching. The extended scheme involves using moments whose orders are higher than kurtosis and generalizes the conventional musical-noise-free method based on kurtosis matching. We formulate standardized-moment matching and explore how effectively the higher-order moments reduce the amount of musical noise. Experimental evaluation results 1) demonstrate that kurtosis matching can reduce musical noise without negatively affecting noise suppression and 2) newly reveal that the sixth-moment matching also achieves low-musical-noise speech enhancement as well as kurtosis matching.

2021-11-01 12:47:10
1 + 3 Twitter

1 0 0 0 Real-Time Full-Band Voice Conversion with Sub-Band Modeling and Data-Driven Phase Estimation of Spectral Differentials

著者: Takaaki SAEKI Yuki SAITO Shinnosuke TAKAMICHI Hiroshi SARUWATARI
出版者: The Institute of Electronics, Information and Communication Engineers
雑誌: IEICE TRANSACTIONS on Information and Systems (ISSN:09168532)
巻号頁・発行日: vol.E104-D, no.7, pp.1002-1016, 2021-07-01
被引用文献数: 2

2021-07-01 08:08:58
1 + 4 Twitter

1 0 0 0 Voice Conversion Using Input-to-Output Highway Networks

著者: Yuki SAITO Shinnosuke TAKAMICHI Hiroshi SARUWATARI
出版者: The Institute of Electronics, Information and Communication Engineers
雑誌: IEICE TRANSACTIONS on Information and Systems (ISSN:09168532)
巻号頁・発行日: vol.E100-D, no.8, pp.1925-1928, 2017-08-01

2017-08-01 00:04:55
1 + 0 Twitter

http://search.ieice.org/bin/summary.php?id=e100-d_8_1925&category=D&year=2017&lang=E&abst=