Korean Singing Voice Synthesis Based on an LSTM Recurrent Neural Network

Juntae Kim, Heejin Choi, Jinuk Park, Minsoo Hahn, Sangjin Kim and Jong-Jin Kim

Abstract:

Singing voice synthesis (SVS) systems generate the singing voice from a musical score. Similar to the text-to-speech synthesis (TTS) field, SVS systems have also been greatly improved since the deep neural network (DNN) framework was introduced. Although they share many parts of the framework, the main difference between TTS and SVS systems is that the feature composing method, between linguistic and musical features, is important for SVS systems. In this paper, we propose a Korean SVS system based on a long-short term memory recurrent neural network (LSTM-RNN). At the feature composing stage, we propose a novel composing method, based on Korean syllable structure. At the synthesis stage, we adopt LSTM-RNN for the SVS. According to our experiments, our composed feature improved the naturalness of the voice, specifically in any part that has to be pronounced for a long time. Furthermore, LSTM-RNN outperformed the DNN based SVS system in both quantitative and qualitative evaluations.

---

Cite as: Kim, J., Choi, H., Park, J., Hahn, M., Kim, S., Kim, J. (2018) Korean Singing Voice Synthesis Based on an LSTM Recurrent Neural Network. Proc. Interspeech 2018, 1551-1555, DOI: 10.21437/Interspeech.2018-1575.

---

BiBTeX Entry:

@inproceedings{Kim2018,
author={Juntae Kim and Heejin Choi and Jinuk Park and Minsoo Hahn and Sangjin Kim and Jong-Jin Kim},
title={Korean Singing Voice Synthesis Based on an LSTM Recurrent Neural Network},
year=2018,
booktitle={Proc. Interspeech 2018},
pages={1551--1555},
doi={10.21437/Interspeech.2018-1575},
url={http://dx.doi.org/10.21437/Interspeech.2018-1575} }