A repository of helpful sources to figure out what the Kotlin compiler really is
The lexer phase breaks source code text into a sequence of lexical tokens:
Parser and PSI/AST
- JetBrains IntelliJ SDK Documentation: Implementing Parser and PSI
- JetBrains Plugins: PSIViewer Library
According to the Kotlin compiler, a PSI, or Program Structure Interface, tree is built on top of the AST, adding semantics and methods for manipulating specific language constructs.
The AST nodes have a direct mapping to text ranges in the underlying document. The bottom-most nodes of the AST match individual tokens returned by the lexer, and higher level nodes match multiple-token fragments. Operations performed on nodes of the AST tree, such as inserting, removing, reordering nodes and so on, are immediately reflected as changes to the text of the underlying document (Implementing Parser and PSI).
The AST is used intensively during semantic analysis, where the compiler checks for correct usage of the elements of the program and the language. It also describes an abstracted representation of what a user writes in Kotlin. AST allows us to change the surface syntax of the language without changing the rest of the compiler (although that rarely happens),