- 著者
-
Shohei Higashiyama
Masao Utiyama
Eiichiro Sumita
Masao Ideuchi
Yoshiaki Oida
Yohei Sakamoto
Isaac Okada
Yuji Matsumoto
- 出版者
- The Association for Natural Language Processing
- 雑誌
- 自然言語処理 (ISSN:13407619)
- 巻号頁・発行日
- vol.27, no.3, pp.499-530, 2020-09-15 (Released:2020-12-15)
- 参考文献数
- 54
- 被引用文献数
-
2
Although limited effort has been devoted to exploring neural models in Japanese word segmentation, much effort has been actively applied to Chinese word segmentation because of the ability to minimize effort in feature engineering. In this work, we propose a character-based neural model that makes joint use of word information useful for disambiguating word boundaries. For each character in a sentence, our model uses an attention mechanism to estimate the importance of multiple candidate words that contain the character. Experimental results show that learning attention to proper words leads to accurate segmentations and that our model achieves better performance than existing statistical and neural models on both in-domain and cross-domain Japanese word segmentation datasets.