Fuzzer: Check failure in toolchain parser "Tried to skip backwards"

[josh11b]

Description of the bug:

Input triggers Check failure in toolchain/parser/parser_context.cpp

What did you do, or what's a simple way to reproduce the bug?

This input triggers the bug:

fn{if[]}

To reproduce:

echo 'fn{if[]}' > /tmp/testcase
bazel run -c dbg //toolchain/driver:carbon -- compile /tmp/testcase

What did you expect to happen?

Diagnostic error report

What actually happened?

Check failure on line https://github.com/carbon-language/carbon-lang/blob/trunk/toolchain/parser/parser_context.cpp#L232 :

CHECK failure at toolchain/parser/parser_context.cpp:232: t >= *position_: Tried to skip backwards from 5 to 4

Any other information, logs, or outputs that you want to share?

Here is the actual output:

Stack dump:
0.      Program arguments: /usr/local/google/home/joshl/.cache/bazel/_bazel_joshl/3df0398fc47b656dd56c4ce9f07c7299/execroot/carbon/bazel-out/k8-dbg/bin/toolchain/driver/carbon compile /tmp/testcase-6561792656474112
1.      Parser stack:
        0.      DeclarationScopeLoop @ 1:1: token 0 : Fn
        1.      FunctionDefinitionFinish @ 1:1: token 0 : Fn
        2.      StatementScopeLoop @ 1:4: token 2 : If
        3.      StatementIfConditionFinish @ 1:4: token 2 : If
        cursor  position_ @ 1:8: token 5 : CloseCurlyBrace
 #0 0x00005589895e378d llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /proc/self/cwd/external/llvm-project/llvm/lib/Support/Unix/Signals.inc:602:11
 #1 0x00005589895e3c7b PrintStackTraceSignalHandler(void*) /proc/self/cwd/external/llvm-project/llvm/lib/Support/Unix/Signals.inc:675:1
 #2 0x00005589895e14b6 llvm::sys::RunSignalHandlers() /proc/self/cwd/external/llvm-project/llvm/lib/Support/Signals.cpp:104:5
 #3 0x00005589895e4975 SignalHandler(int) /proc/self/cwd/external/llvm-project/llvm/lib/Support/Unix/Signals.inc:413:1
 #4 0x00007f1e8b28a540 (/lib/x86_64-linux-gnu/libc.so.6+0x3c540)
 #5 0x00007f1e8b2d812c __pthread_kill_implementation ./nptl/./nptl/pthread_kill.c:44:76
 #6 0x00007f1e8b28a4a2 raise ./signal/../sysdeps/posix/raise.c:27:6
 #7 0x00007f1e8b2744b2 abort ./stdlib/./stdlib/abort.c:81:7
 #8 0x00005589890e8585 (/usr/local/google/home/joshl/.cache/bazel/_bazel_joshl/3df0398fc47b656dd56c4ce9f07c7299/execroot/carbon/bazel-out/k8-dbg/bin/toolchain/driver/carbon+0x56e9585)
 #9 0x0000558985fc5405 (/usr/local/google/home/joshl/.cache/bazel/_bazel_joshl/3df0398fc47b656dd56c4ce9f07c7299/execroot/carbon/bazel-out/k8-dbg/bin/toolchain/driver/carbon+0x25c6405)
#10 0x00005589890ae4a8 Carbon::ParserContext::SkipTo(Carbon::TokenizedBuffer::Token) /proc/self/cwd/toolchain/parser/parser_context.cpp:232:3
#11 0x00005589890ae19b Carbon::ParserContext::ConsumeAndAddCloseSymbol(Carbon::TokenizedBuffer::Token, Carbon::ParserContext::StateStackEntry, Carbon::ParseNodeKind) /proc/self/cwd/toolchain/parser/parser_context.cpp:109:5
#12 0x00005589890c1096 Carbon::ParserHandleParenConditionFinishAsIf(Carbon::ParserContext&) /proc/self/cwd/toolchain/parser/parser_handle_paren_condition.cpp:38:1
#13 0x00005589890a39ff Carbon::ParseTree::Parse(Carbon::TokenizedBuffer&, Carbon::DiagnosticConsumer&, llvm::raw_ostream*) /proc/self/cwd/./toolchain/parser/parser_state.def:480:1
#14 0x0000558985fbd8f5 Carbon::Driver::Compile(Carbon::Driver::CompileOptions const&) /proc/self/cwd/toolchain/driver/driver.cpp:426:21
#15 0x0000558985fbd16f Carbon::Driver::RunCommand(llvm::ArrayRef<llvm::StringRef>) /proc/self/cwd/toolchain/driver/driver.cpp:341:14
#16 0x0000558985fbb826 main /proc/self/cwd/toolchain/driver/driver_main.cpp:33:25
#17 0x00007f1e8b2756ca __libc_start_call_main ./csu/../sysdeps/nptl/libc_start_call_main.h:74:3
#18 0x00007f1e8b275785 call_init ./csu/../csu/libc-start.c:128:20
#19 0x00007f1e8b275785 __libc_start_main ./csu/../csu/libc-start.c:347:5
#20 0x0000558985fbb5e1 _start (/usr/local/google/home/joshl/.cache/bazel/_bazel_joshl/3df0398fc47b656dd56c4ce9f07c7299/execroot/carbon/bazel-out/k8-dbg/bin/toolchain/driver/carbon+0x25bc5e1)
CHECK failure at toolchain/parser/parser_context.cpp:232: t >= *position_: Tried to skip backwards from 5 to 4

[Jon Ross-Perkins]

I'm looking at a related issue which will fix this. For reference, I think in #3064 where it does *(TokenizedBuffer::TokenIterator(state.token) + 1) is probably causing this because it allows for nodes to be added based on unconsumed tokens, and it's generically handling open tokens rather than explicitly parentheses (which it's intended to handle open parens).