Tag Archives: Chinese Word Segmenter

Open Source Text Processing Project: Jieba

Posted on by January 13, 2016
Reply

Jieba： Chinese text segmentation Project Website: None Github Link: https://github.com/fxsjy/jieba Description “Jieba” (Chinese for “to stutter”) Chinese text segmentation: built to be the best Python Chinese word segmentation module. Features Support three types of segmentation mode: Accurate Mode attempts to … Continue reading →

Open Source Text Processing Project: Stanford Word Segmenter

Posted on by December 31, 2015
Reply

Stanford Word Segmenter Project Website: http://nlp.stanford.edu/software/segmenter.shtml Github Link: None Description Tokenization of raw text is a standard pre-processing step for many NLP tasks. For English, tokenization usually involves punctuation splitting and separation of some affixes like possessives. Other languages require … Continue reading →