pvdz / tenko

An 100% spec compliant ES2021 JavaScript parser written in JS

Home Page:https://pvdz.github.io/tenko/repl

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Incorrectly accepts `async ({await}) => 1`

bakkot opened this issue · comments

When CoverCallExpressionAndAsyncArrowHead is refined to async ArrowFormalParameters, the ArrowFormalParameters nonterminal is given the flag [+Await], and await is only a valid IdentifierReference when the flag is -Await. So async ({await}) => 1 should be a parse error. But it isn't. Instead the await is treated as an identifier.

(Also, congrats on the parser!)

Thanks for reporting it. I'll be sure to make this work soon :)

This turns out more difficult than I thought.

The problem is a sibling case:

async({await});
async({await}) => x;

The call is valid unless module goal, the arrow is never valid.

The problem here lies in the way that I'm handling the "are we parsing async code"-state. In a nutshell either we are or we aren't parsing async code. This is fine for most cases, except this one, where we are parsing code that "might be async".

The problem is in _parseGroupToplevels, where we receive the asyncToken param which tells us whether the group was prefixed with async. At this point we won't know whether we are parsing the call or the arrow, yet, as we can only know this once we find the arrow. That's a general arrow problem, and async has some additional edge cases like this.

While parsing the parser mainly tracks whether or not the parsed content might be destructible, must be destructible, cannot be destructible, or can only be destructible through assignment.

If the code is async then a variable named await will trigger an error regardless, so that error is not conditional (nor is there a way to return this in its destructibility). If the code is not async then it won't trigger an error, whether call or arrow. Unfortunately for the case in this issue we need some sort of "maybe async" state, which is the only one of its kind... :/

Just writing this out to get a better picture of the problem, and explain why it's not a nobrainer to solve :)

Actually, async(await)=>x is properly handled so I did already solve this. And that's through the PIGGY_BACK_SAW_AWAIT_KEYWORD state. So the fix here is to make sure object/array parsers also properly reflect these await/yield states. Okay :)

The full list of await-in-arrow cases is this:

// Without async modifier:
// Await expressions are taken care of immediately by lexerFlags, all other cases are valid
// [v]: `await => x`
// [v]: `(await) => x`
// [v]: `(...await) => x`
// [v]: `(a = await) => x`
// [x]: `(a = await b) => x`
// [v]: `({await}) => x`
// [v]: `({a = await}) => x`
// [x]: `({a = await b}) => x`
// [v]: `({a: b = await}) => x`
// [x]: `({a: b = await c}) => x`
// [v]: `([a = await]) => x`
// [x]: `([a = await b]) => x`

// No modifiers but living in async space (makes await expressions legal, exposes cases where we do need checks)
// If async space, arrow, and await piggy --> error
// [x]: `async () => await => x`
// [x]: `async () => (await) => x`
// [x]: `async () => (...await) => x`
// [x]: `async () => (a = await) => x`
// [x]: `async () => (a = await b) => x`
// [x]: `async () => ({await}) => x`
// [x]: `async () => ({a = await}) => x`
// [x]: `async () => ({a = await b}) => x`
// [x]: `async () => ({a: b = await}) => x`
// [x]: `async () => ({a: b = await c}) => x`
// [x]: `async () => ([a = await]) => x`
// [x]: `async () => ([a = await b]) => x`

// Same cases and now with `async` modifier
// If arrow and keyword --> error
// [x]: `async await => x`
// [x]: `async (await) => x`
// [x]: `async (...await) => x`
// [x]: `async (a = await) => x`
// [x]: `async (a = await b) => x`
// [x]: `async ({await}) => x`
// [x]: `async ({a = await}) => x`
// [x]: `async ({a = await b}) => x`
// [x]: `async ({a: b = await}) => x`
// [x]: `async ({a: b = await c}) => x`
// [x]: `async ([a = await]) => x`
// [x]: `async ([a = await b]) => x`

// The next two sets are the async-call variants. We don't have to check the piggy for them because
// they don't trigger the param checks, of course. So we these cases get picked by regular checks. Yay.

// The legacy async call variants
// [v]: `async (await)`
// [v]: `async (...await)`
// [v]: `async (a = await)`
// [x]: `async (a = await b)`
// [v]: `async ({await})`
// [v]: `async ({a = await})`
// [x]: `async ({a = await b})`
// [v]: `async ({a: b = await})`
// [x]: `async ({a: b = await c})`
// [v]: `async ([a = await])`
// [x]: `async ([a = await b])`

// Legacy calls wrapped in an async function
// [x]: `async x => async (await)`
// [x]: `async x => async (...await)`
// [x]: `async x => async (a = await)`
// [v]: `async x => async (a = await b)`
// [x]: `async x => async ({await})`
// [v]: `async x => async ({a = await})`
// [x]: `async x => async ({a: b = await})`
// [v]: `async x => async ([a = await])`

They are now all parsing properly. And I was able to eliminate a redundant flag (used to return whether the arrow header contained an await expression or varname, now just whether it contained await at all).

Thanks for reporting! Sorry it took so long to fix.

Please let me know if you can think of anything else.

(The shorthand object property was not being checked for await at all, btw, so that was another bug)