asottile / tokenize-rt

A wrapper around the stdlib `tokenize` which roundtrips.

asottile/tokenize-rt Issues

Umlaut in triple quotes breaks roundtrip
Closed 9 months ago1
Missing py.typed
Closed a year ago5
Broken handling of unicode f literals
Closed 2 years ago4
Consider creating man page for tokenize-rt executable
Closed 2 years ago1
Why not send upstream?
Closed 3 years ago3
empty newline at end of file produces incorrect utf8_byte_offset
Closed 3 years ago
Produce offset information for extra tokens
Closed 3 years ago
Would you like property-based tests?
Closed 3 years ago2
Does not produce `ESCAPED_NL` token for escaped `\r\n`
Closed 7 years ago1
Consider emitting an `ESCAPED_NL` token
Closed 7 years ago