Glyce: Glyph-vectors for Chinese Character Representations

by   Wei Wu, et al.

It is intuitive that NLP tasks for logographic languages like Chinese should benefit from the use of the glyph information in those languages. However, due to the lack of rich pictographic evidence in glyphs and the weak generalization ability of standard computer vision models on character data, an effective way to utilize the glyph information remains to be found. In this paper, we address this gap by presenting the Glyce, the glyph-vectors for Chinese character representations. We make three major innovations: (1) We use historical Chinese scripts (e.g., bronzeware script, seal script, traditional Chinese, etc) to enrich the pictographic evidence in characters; (2) We design CNN structures tailored to Chinese character image processing; and (3) We use image-classification as an auxiliary task in a multi-task learning setup to increase the model's ability to generalize. For the first time, we show that glyph-based models are able to consistently outperform word/char ID-based models in a wide range of Chinese NLP tasks. Using Glyce, we are able to achieve the state-of-the-art performances on 13 (almost all) Chinese NLP tasks, including (1) character-Level language modeling, (2) word-Level language modeling, (3) Chinese word segmentation, (4) name entity recognition, (5) part-of-speech tagging, (6) dependency parsing, (7) semantic role labeling, (8) sentence semantic similarity, (9) sentence intention identification, (10) Chinese-English machine translation, (11) sentiment analysis, (12) document classification and (13) discourse parsing


page 1

page 2

page 3

page 4


Glyph-aware Embedding of Chinese Characters

Given the advantage and recent success of English character-level and su...

Is Word Segmentation Necessary for Deep Learning of Chinese Representations?

Segmenting a chunk of text into words is usually the first step of proce...

Radical-level Ideograph Encoder for RNN-based Sentiment Analysis of Chinese and Japanese

The character vocabulary can be very large in non-alphabetic languages s...

Building a Kannada POS Tagger Using Machine Learning and Neural Network Models

POS Tagging serves as a preliminary task for many NLP applications. Kann...

Investigating Glyph Phonetic Information for Chinese Spell Checking: What Works and What's Next

While pre-trained Chinese language models have demonstrated impressive p...

Character-based Neural Networks for Sentence Pair Modeling

Sentence pair modeling is critical for many NLP tasks, such as paraphras...

Please sign up or login with your details

Forgot password? Click here to reset