著者

Shohei Higashiyama Masao Utiyama Eiichiro Sumita Masao Ideuchi Yoshiaki Oida Yohei Sakamoto Isaac Okada Yuji Matsumoto

出版者

The Association for Natural Language Processing

雑誌

自然言語処理 (ISSN:13407619)

巻号頁・発行日

vol.27, no.3, pp.499-530, 2020-09-15 (Released:2020-12-15)

参考文献数

54

被引用文献数

2

---

Although limited effort has been devoted to exploring neural models in Japanese word segmentation, much effort has been actively applied to Chinese word segmentation because of the ability to minimize effort in feature engineering. In this work, we propose a character-based neural model that makes joint use of word information useful for disambiguating word boundaries. For each character in a sentence, our model uses an attention mechanism to estimate the importance of multiple candidate words that contain the character. Experimental results show that learning attention to proper words leads to accurate segmentations and that our model achieves better performance than existing statistical and neural models on both in-domain and cross-domain Japanese word segmentation datasets.