microsoft / MIEngine

Bug type: Debugger

Describe the bug

OS and Version: Linux (arm64 and x86_64), gdb 9.2 (NOTE: does not reproduce on macOS with lldb)
VS Code Version: vimspector
C/C++ Extension Version: 1.9.7
Other extensions you installed (and if the issue persists after disabling them): N/A
A clear and concise description of what the bug is.

After upgrading vimspector to 1.9.7 vscode-cpptools I'm getting regression test failures. The first one I have diagnosed as a MIEngine exception.

To Reproduce

There appears to be timing involved because manual steps do not reproduce, however my automated tests fail reliably. The test in question does this:

Open this file: https://github.com/puremourning/vimspector/blob/master/tests/testdata/cpp/simple/simple.cpp
Start debugging with stopAtEntry true
Execution pauses at line 15
Set a breakpoint at line 16 and ensure it goes 'VERIFIED'
Delete the breakpoint
add it again
Continue execution
Expect a stopped event on line 16.

However on both linux x86_64 and linux arm64, I get the following Unable to continue. Operation failed with error code 0x80004004 preceded by ERROR: Internal error in MIEngine. Exception of type 'System.NullReferenceException' was thrown.

2022-03-26 14:09:32,834 - DEBUG - Sending Message: {"command": "setBreakpoints", "arguments": {"source": {"name": "simple.cpp", "path": "/home/dev/vimspector/tests/testdata/cpp/simple/simple.cpp"}, "breakpoints": [{"line": 16}], "sourceModified": false}, "seq": 18, "type": "request"}
2022-03-26 14:09:32,834 - DEBUG - Sending Message: {"command": "setFunctionBreakpoints", "arguments": {"breakpoints": []}, "seq": 19, "type": "request"}
2022-03-26 14:09:32,835 - DEBUG - Sending Message: {"command": "setExceptionBreakpoints", "arguments": {"filters": []}, "seq": 20, "type": "request"}
2022-03-26 14:09:32,847 - DEBUG - Message received: {'type': 'response', 'request_seq': 18, 'success': True, 'command': 'setBreakpoints', 'body': {'breakpoints': [{'id': 2, 'verified': True, 'line': 16, 'BoundBreakpoints': []}]}, 'seq': 41}
2022-03-26 14:09:32,847 - DEBUG - Message received: {'type': 'event', 'event': 'breakpoint', 'body': {'reason': 'changed', 'breakpoint': {'id': 2, 'verified': True, 'line': 16, 'BoundBreakpoints': []}}, 'seq': 42}
2022-03-26 14:09:32,848 - DEBUG - Message received: {'type': 'response', 'request_seq': 19, 'success': True, 'command': 'setFunctionBreakpoints', 'body': {'breakpoints': []}, 'seq': 43}
2022-03-26 14:09:32,848 - DEBUG - Message received: {'type': 'response', 'request_seq': 20, 'success': True, 'command': 'setExceptionBreakpoints', 'body': {}, 'seq': 44}
2022-03-26 14:09:32,849 - DEBUG - Sending Message: {"command": "continue", "arguments": {"threadId": 416}, "seq": 21, "type": "request"}
2022-03-26 14:09:32,884 - DEBUG - Message received: {'type': 'event', 'event': 'output', 'body': {'category': 'stderr', 'output': "ERROR: Internal error in MIEngine. Exception of type 'System.NullReferenceException' was thrown.\r\n\r\n   at Microsoft.MIDebugEngine.DebuggedProcess.Execute(DebuggedThread thread)\n   at Microsoft.MIDebugEngine.WorkerThread.TrySetOperationInternal(Delegate op)\n   at Microsoft.MIDebugEngine.WorkerThread.SetOperationInternal(Delegate op)\n   at Microsoft.MIDebugEngine.WorkerThread.RunOperation(AsyncOperation op)\n   at Microsoft.MIDebugEngine.AD7Engine.Continue(IDebugThread2 pThread)\n"}, 'seq': 45}
2022-03-26 14:09:32,887 - DEBUG - Message received: {'type': 'response', 'request_seq': 21, 'success': False, 'command': 'continue', 'message': 'Unable to continue. Operation failed with error code 0x80004004.', 'body': {}, 'seq': 46}
2022-03-26 14:09:32,888 - ERROR - Request failed (unhandled): Unable to continue. Operation failed with error code 0x80004004.

the configuration being used is captured in the initialise and launch requests:

2022-03-26 14:09:21,136 - DEBUG - Sending Message: {"command": "initialize", "arguments": {"adapterID": "cppdbg", "clientID": "vimspector", "clientName": "vimspector", "linesStartAt1": true, "columnsStartAt1": true, "locale": "en_GB", "pathFormat": "path", "supportsVariableType": true, "supp
ortsVariablePaging": false, "supportsRunInTerminalRequest": true, "supportsMemoryReferences": true}, "seq": 0, "type": "request"}
2022-03-26 14:09:21,791 - DEBUG - Message received: {'type': 'response', 'request_seq': 0, 'success': True, 'command': 'initialize', 'body': {'supportsConfigurationDoneRequest': True, 'supportsFunctionBreakpoints': True, 'supportsConditionalBreakpoints': True, 'supportsEvaluateForHovers': Tr
ue, 'exceptionBreakpointFilters': [{'filter': 'all', 'label': 'All C++ Exceptions', 'default': False, 'supportsCondition': True, 'conditionDescription': 'std::out_of_range,std::invalid_argument'}], 'supportsSetVariable': True, 'supportsGotoTargetsRequest': True, 'supportsCompletionsRequest':
 True, 'completionTriggerCharacters': [], 'supportsModulesRequest': True, 'supportedChecksumAlgorithms': [], 'supportsValueFormattingOptions': True, 'supportsLogPoints': True, 'supportsDataBreakpoints': True, 'supportsReadMemoryRequest': True, 'supportsDisassembleRequest': True, 'supportsCli
pboardContext': True, 'supportsSteppingGranularity': True, 'supportsInstructionBreakpoints': True, 'supportsExceptionFilterOptions': True}, 'seq': 1}
2022-03-26 14:09:21,791 - DEBUG - Sending Message: {"command": "launch", "arguments": {"args": [], "cwd": "/home/dev/vimspector/tests/testdata/cpp/simple", "environment": [], "type": "cppdbg", "targetArchitecture": "x86_64", "request": "launch", "program": "/home/dev/vimspector/tests/testdat
a/cpp/simple/simple", "externalConsole": false, "stopAtEntry": true, "stopOnEntry": true, "MIMode": "gdb", "name": "test"}, "seq": 1, "type": "request"}

Additional context

Gist of the full vimspector log which includes DAP trace with the engine Logging is here: https://gist.github.com/puremourning/f52268564824edbdf6779307ce1b88b0

Apologies if this should be raised with MIEngine...

There appears to be timing involved because manual steps do not reproduce,

I also tried it out by hand and was unable to reproduce it, however one thing VS Code does is that it sends a request and waits for a response, but it could be interrupted by multiple events.

In your test case, it looks like its sending setBreakpoints, setFunctionBreakpoints, and setExceptionBreakpoints at the same time, which is throwing off the engine.

The change that potentially be the issue from 1.7.1 to 1.9.1 is the exception breakpoint support.

Do you have a way for me to run that specific test so I can figure out the NPE or is it possible for you to collect a dump for OpenDebugAD7 with dotnet-dump?

Sure, to run the test, you can:

git clone https://github.com/puremourning/vimspector to some directory
docker pull puremourning/vimspector:manual-x86_64
it's easiest to run in the container - ./tests/manual/run (if on Mac or linux) - on windows you'll probably have to do the equivalent, i.e. something like docker run -cap-add=SYS_PTRACE --security-opt seccomp=unconfined -it puremourning/vimspector:manual-x86_64 -v /path/to/vimspector:/home/dev/vimspector
cd /home/dev/vimspector
./run_tests --install breakpoints.test.vim:Test_DisableBreakpointWhileDebugging\(

More info: https://github.com/puremourning/vimspector/blob/master/CONTRIBUTING.md#running-the-tests-locally

is it possible for you to collect a dump for OpenDebugAD7 with dotnet-dump?

I'm happy to try this, but I don't really know what to do :)

I'll run the instructions provided above on Monday to see if can get the issue.

is it possible for you to collect a dump for OpenDebugAD7 with dotnet-dump?

I'm happy to try this, but I don't really know what to do :)

I think this is the easiest way to do it unless you can get the automatic crash dump working by enabling
COMPlus_DbgEnableMiniDump=1 in your environment but I haven't been able to do so.

See Configure createdump to run at process termination

The "easiest" way might be.

Install dotnet-dump
Use the VS Code C# extension to attach to the OpenDebugAD7 process during test startup.
When the VS Code C# extension stops hits the exception, find the PID of OpenDebugAD7 and run dotnet-dump collect -p <PID> and share the dump.

edit: Found it under test-base/gadgets

@puremourning Where are the extensions loaded? To debug the process I was going to modify the extension's js code to halt the debugger adapter for a debugger to attach to it.

it's easiest to run in the container - ./tests/manual/run (if on Mac or linux) - on windows you'll probably have to do the equivalent, i.e. something like docker run -cap-add=SYS_PTRACE --security-opt seccomp=unconfined -it puremourning/vimspector:manual-x86_64 -v /path/to/vimspector:/home/dev/vimspector

FYI for other Windows users, its easier to just create the docker container and clone because of \r\n line endings on Windows having issues with the scripts.

I've narrowed it down to

await ExceptionManager.EnsureSettingsUpdated();

Reference:

MIEngine/src/MIDebugEngine/Engine.Impl/DebuggedProcess.cs

Line 1674 in 11fe381

await ExceptionManager.EnsureSettingsUpdated();

Somwhere in EnsureSettingsUpdated is throwing a NPE and this must have come in with the exception breakpoint work we added.

Still need to investigate further and migrating this to MIEngine repo.

@puremourning Thanks for reporting this issue! Looks like a lucky timing issue that was caught by your tests.

Great. Thanks for looking into it.

Will be fixed in v1.10.0-insiders

System.NullReferenceException when continuing after adding breakpoint.