Gnuplot child application not spawned properly on Windows

[scimax]

Hello,
I am trying to use the gnuplot kernel on Windows. After fixing the issue of the permanently dying kernel the code in a notebook is still not executed. Now I think I have tracked down the issue to the metakernel. As I'm not sure I appreciate any hints!

To examine the behavior of the repl wrapper and pexpect I imported the modules and ran them manually. With cmd it works nicely, like the cmd() function in replwrap.py:

from metakernel import replwrap
import re
cmd_c=replwrap.REPLWrapper("cmd", re.compile(r"[A-Z]:\\.*>"), None, echo=True)
cmd_c.run_command("echo 'Hello'")  # outputs: "'Hello'\r\n\r\n"

The same works with powershell instead of cmd.

If I try the same with gnuplot the cell runs and keeps running infinitely.
So I can't even send a command to the gnuplot instance via run_command(...). But gnuplot started as I observed it in the task manager.
To start gnuplot I tried

gnu_c=replwrap.REPLWrapper("gnuplot", re.compile(r".*>"), None, echo=True)

and

gnu_c=replwrap.REPLWrapper(child, re.compile(r"gnuplot>"), None, echo=True)

What I figured out is that gnuplot outputs Encoding set to 'cp1252' when executing it in interactive mode in the terminal (Powershell). But the REPLWrapper spawns a process with encoding='utf-8'. Could that cause an issue like that?

Originally I started with a jupyter notebook with the gnuplot kernel. The console logs of the notebook showed a warning entry of Timeout waiting for kernel_info reply from a0580037-7426-4d3e-abd2-5a426e21d3d5. I got the same when I started a jupyter console with jupyter console --kernel gnuplot:

RuntimeError: Kernel didn't respond to kernel_info_request

But in my opinion this exception is only raised because the initialization of the REPLWrapper is still running.

System details:

• Windows 10, Version 1803
• Python 3.6.4
• jupyter 4.4.0
• metakernel 0.20.14
• gnuplot_kernel 0.2.3 (dev version from github)
• pexpect 4.6.0

[Douglas Blank]

Thanks for the report. Personally, I don't have time to look at this right now, but I will note that I had a similar issue when trying to replwrap an older java9 jshell console program. There was a weird control code in the interaction, and caused the wrap to fail.

[scimax]

Thanks for your quick reply! Do you have any idea how to get more output from the spawned process?

Somehow, I get the same issue when I try Julia instead of gnuplot as the command argument. I know that there is a separate kernel for Julia. I was just wondering.

[Steven Silvester]

Hi @scimax, we could add a stream_handler to capture individual lines, but that doesn't help at startup. You could set it to sys.stdout.write here.

[scimax]

Thanks, @blink1073, but self._stream_handler = sys.stdout.write did not output more information.

But I changed the timeout in the _expect_prompt() (here) to 10 and other values. Before that line I also added a print statement: print(expects)

With cmd as the command in the REPLWrapper the output is as expected:

cmd_c=replwrap.REPLWrapper("cmd", re.compile(r".*>"), None, echo=True)
ignore
[re.compile('.*>'), 'PEXPECT_PROMPT_', 'PEXPECT_PROMPT+', '\r\n']
Microsoft Windows [Version 10.0.17134.165][re.compile('.*>'), 'PEXPECT_PROMPT_', 'PEXPECT_PROMPT+', '\r\n']
(c) 2018 Microsoft Corporation. All rights reserved.[re.compile('.*>'), 'PEXPECT_PROMPT_', 'PEXPECT_PROMPT+', '\r\n']
[re.compile('.*>'), 'PEXPECT_PROMPT_', 'PEXPECT_PROMPT+', '\r\n']
>>> exit()

With gnuplot I get a Timeout exception now, so I get the impression now that the error is located in the pexpect package.

>>> gnu_c=replwrap.REPLWrapper("gnuplot", re.compile(r".*>"), None, echo=True)
ignore
[re.compile('.*>'), 'PEXPECT_PROMPT_', 'PEXPECT_PROMPT+', '\r\n']
Traceback (most recent call last):
 File "C:\Python36\lib\site-packages\pexpect\expect.py", line 109, in expect_loop
   return self.timeout()
 File "C:\Python36\lib\site-packages\pexpect\expect.py", line 82, in timeout
   raise TIMEOUT(msg)
pexpect.exceptions.TIMEOUT: <pexpect.popen_spawn.PopenSpawn object at 0x0000027E0A95B748>
searcher: searcher_re:
   0: re.compile('.*>')
   1: re.compile('PEXPECT_PROMPT_')
   2: re.compile('PEXPECT_PROMPT+')
   3: re.compile('\r\n')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "C:\Python36\lib\site-packages\metakernel\replwrap.py", line 101, in __init__
   self._expect_prompt()
 File "C:\Python36\lib\site-packages\metakernel\replwrap.py", line 131, in _expect_prompt
   pos = self.child.expect(expects, timeout=10)
 File "C:\Python36\lib\site-packages\pexpect\spawnbase.py", line 341, in expect
   timeout, searchwindowsize, async_)
 File "C:\Python36\lib\site-packages\pexpect\spawnbase.py", line 369, in expect_list
   return exp.expect_loop(timeout)
 File "C:\Python36\lib\site-packages\pexpect\expect.py", line 119, in expect_loop
   return self.timeout(e)
 File "C:\Python36\lib\site-packages\pexpect\expect.py", line 82, in timeout
   raise TIMEOUT(msg)
pexpect.exceptions.TIMEOUT: <pexpect.popen_spawn.PopenSpawn object at 0x0000027E0A95B748>
searcher: searcher_re:
   0: re.compile('.*>')
   1: re.compile('PEXPECT_PROMPT_')
   2: re.compile('PEXPECT_PROMPT+')
   3: re.compile('\r\n')
<pexpect.popen_spawn.PopenSpawn object at 0x0000027E0A95B748>
searcher: searcher_re:
   0: re.compile('.*>')
   1: re.compile('PEXPECT_PROMPT_')
   2: re.compile('PEXPECT_PROMPT+')
   3: re.compile('\r\n')

[scimax]

Does someone have an idea how to fix that issue?