LexicalAnalyzer This program converts the text in a file into sequence of tokens which can be recognized by the lexical analyzer.