Project Website: https://code.google.com/archive/p/berkeleylm/
Github Link: https://github.com/adampauls/berkeleylm
An N-gram Language Model Library from UC Berkeley
This project provides a library for estimating storing large n-gram language models in memory and accessing them efficiently. It is described in http://nlp.cs.berkeley.edu/pubs/Pauls-Klein_2011_LM_paper.pdf’>this paper. Its data structures are faster and smaller than http://nlp.cs.berkeley.edu/’>SRILM and nearly as fast as http://kheafield.com/code/kenlm/’>KenLM despite being written in Java instead of C++. It also achieves the best published lossless encoding of the Google n-gram corpus.