ULS is an library for general purpose lexical analysis, with support for UTF-8. It comes with C/C++ libraries and a couple of tools for Linux and Windows. It's a intuitive, practical, flexible, and optimized tokenizer. ULS can instantiate multiple objects for lexical analyses. The objects can process multiple (nested) inputs of different languages. The language specification is specified in configuration file suffixed by *.ulc. ULS can tokenize the input file, which encoded by UTF-8. The input files may contain the words in a localized language as identifiers. ULS can stream the tokens from many input files to another output (files).The stream can be stored in a *.uls file and replayed from it whenever necessary.
