listOf, nonemptyListOf and emptyListOf handling in toAST
charlespwd opened this issue · comments
Docs specify that, by default, array-like things are getting transformed into arrays. This is true of ListOf but not of the lexical version listOf
.
I had to add the following to my mappings object for my list of things to be turned into a list.
listOf: 0,
nonemptyListOf(first, sep, rest) {
return [first.toAST(this.args.mapping)].concat(rest.toAST(this.args.mapping));
},
emptyListOf() {
return [];
},
I suspect that these should be added to the defaults?
Perhaps here ?
I started implementing a fix for this, but then I remembered the reason that we don't have defaults for the lexical versions of these rules —
One of the assumptions that toAST
makes is that lexical rules represent tokens — so the default handling of lexical rules is to just return a string containing the input text that was consumed. That's a reasonable assumption for most lexical rules, but I think that listOf
and friends are an exception. I think in most of the places where you'd use the lexical listOf
, it's much more useful to discard the separator. E.g.: slash-separate paths (path = listOf<pathEl, "/">
), TSV/CSV formats (line = listOf<col, "\t">
, etc.
So, I think we should change the defaults here, but I wanted to record my thinking for posterity :-)
This has been fixed on main. However, since it's a breaking change, it won't be released until the next major version (v17), probably in the next few weeks.