can write custom token prossessing pipeline
fergiemcdowall opened this issue · comments
Fergus McDowall commented
Once string is tokenized, it should be possible to write stages that manipulate individual tokens (upcase, lowcase, substitution, etc...)