On a crash of a command, the next command in pipe should not hang on closed pipe
kaby76 opened this issue · comments
In a piped command, e.g., trparse fu | trtree, if the first command (trparse) fails with a crash, the next command should see and EOF and complete. Currently, it hangs. I'm not sure why it is doing this. In C/C++ this would never happen--the file is closed and the next program sees that. For some reason C# is different.
Really annoying behavior programmed by MS. Due to crashes in regression tests, GitHub Actions hangs with pipelined commands like this:
trparse *.g4 | trxgrep ' // 4 4 4' | trtree
Adding debugging output around the catch code in trxgrep.
public static void Main(string[] args)
{
try
{
new Program().MainInternal(args);
}
catch (Exception e)
{
System.Console.Error.WriteLine(e.ToString());
System.Console.Error.WriteLine("Exiting trxgrep");
System.Environment.Exit(1);
System.Console.Error.WriteLine("(after)");
}
}
Over in the reader trtree, we are stuck on a ReadToEnd() call--it never receives the close pipe.
{
string lines = null;
if (!(config.File != null && config.File != ""))
{
if (config.Verbose)
{
System.Console.Error.WriteLine("reading from stdin");
}
for (; ; )
{
lines = System.Console.In.ReadToEnd();
if (lines != null && lines != "") break;
}
}
else ...
Output:
$ trparse *.g4 | trxgrep ' // 4 4 4' | trtree
line 1:6 mismatched input '4' expecting <EOF>
org.eclipse.wst.xml.xpath2.processor.XPathParserException: ANTLR parser error
at org.eclipse.wst.xml.xpath2.processor.internal.InternalXPathParser.parse(String xpath, Boolean isRootlessAccess) in C:\Users\Kenne\Documents\GitHub\Domemtech.Trash\src\AntlrTreeEditing\org\eclipse\wst\xml\xpath2\processor\internal\InternalXPathParser.cs:line 79
at org.eclipse.wst.xml.xpath2.processor.XPathParserInAntlr.parse(String xpath) in C:\Users\Kenne\Documents\GitHub\Domemtech.Trash\src\AntlrTreeEditing\org\eclipse\wst\xml\xpath2\processor\XPathParserInAntlr.cs:line 38
at org.eclipse.wst.xml.xpath2.processor.Engine.parseExpression(String expression, StaticContext context) in C:\Users\Kenne\Documents\GitHub\Domemtech.Trash\src\AntlrTreeEditing\org\eclipse\wst\xml\xpath2\processor\Engine.cs:line 34
at Trash.Command.Execute(Config config) in C:\Users\Kenne\Documents\GitHub\Domemtech.Trash\src\trxgrep\Command.cs:line 95
at Trash.Program.MainInternal(String[] args) in C:\Users\Kenne\Documents\GitHub\Domemtech.Trash\src\trxgrep\Program.cs:line 70
at Trash.Program.Main(String[] args) in C:\Users\Kenne\Documents\GitHub\Domemtech.Trash\src\trxgrep\Program.cs:line 14
Exiting trxgrep
Fix attempt # 1
public static void Main(string[] args)
{
try
{
new Program().MainInternal(args);
}
catch (Exception e)
{
System.Console.Error.WriteLine(e.ToString());
System.Console.Error.WriteLine("Exiting trxgrep");
System.Environment.ExitCode = 1;
System.Console.Error.WriteLine("(after)");
}
}
=> Doesn't work -- trtree still hangs.
Fix attempt # 2
public static void Main(string[] args)
{
try
{
new Program().MainInternal(args);
}
catch (Exception e)
{
System.Console.Error.WriteLine(e.ToString());
System.Console.Error.WriteLine("Exiting trxgrep");
System.Console.Error.WriteLine("(after)");
}
}
=> Still hangs. I'm wondering if the pipes are ever closed at all due to error handling and garbage collection.
Fix attempt # 3
public static void Main(string[] args)
{
try
{
var bullshit = new Program();
bullshit.MainInternal(args);
}
catch (Exception e)
{
System.Console.Error.WriteLine(e.ToString());
System.Console.Error.WriteLine("Exiting trxgrep");
System.Console.Error.WriteLine("(after)");
}
}
=> still hangs. I wonder if I need to write something to stdout.
Attempt # 4
public static void Main(string[] args)
{
try
{
System.Console.Write("");
var bullshit = new Program();
bullshit.MainInternal(args);
}
catch (Exception e)
{
System.Console.Error.WriteLine(e.ToString());
System.Console.Error.WriteLine("Exiting trxgrep");
System.Console.Error.WriteLine("(after)");
}
}
=> Hangs.
Attempt # 5 -- Use "Out" explicitly, and flush.
public static void Main(string[] args)
{
try
{
var bullshit = new Program();
bullshit.MainInternal(args);
}
catch (Exception e)
{
System.Console.Error.WriteLine(e.ToString());
System.Console.Error.WriteLine("Exiting trxgrep");
System.Console.Error.WriteLine("(after)");
System.Console.Out.Write("");
System.Console.Out.Flush();
}
}
=> Still hangs. But where??
Attempt # 6 -- Change reader, probably stuck in loop.
if (!(config.File != null && config.File != ""))
{
if (config.Verbose)
{
System.Console.Error.WriteLine("reading from stdin");
}
for (; ; )
{
System.Console.Error.WriteLine("Start loop");
lines = System.Console.In.ReadToEnd();
System.Console.Error.WriteLine("After ReadToEnd");
if (lines != null && lines != "")
{
System.Console.Error.WriteLine("Breaking");
break;
}
}
}
=> Stuck in loop. We are back with this issue: dotnet/runtime#50780
Yes, this is a reader/writer problem described here.
Writer:
namespace w
{
class Program
{
static void Main(string[] args)
{
int count = System.Int32.Parse(args[0]);
System.Console.Error.WriteLine("Parse completed of C:\\Users\\kenne\\Documents\\AntlrVSIX\\test\\A.g4");
for (int j = 0; j < count; ++j)
{
var s = @"{
""FileName"": ""C:\\Users\\kenne\\Documents\\AntlrVSIX\\test\\A.g4""
}";
System.Console.WriteLine(s);
}
}
}
}
Reader:
using System;
using System.Collections.Generic;
namespace r
{
class Program
{
static void Main(string[] args)
{
try {
//bool first = true;
List<string> lines = new List<string>();
System.Console.Error.WriteLine("reading...");
var s = System.Console.In.ReadToEnd();
if (s == null) System.Console.Error.WriteLine("null read");
else if (s == "") System.Console.Error.WriteLine("read empty string");
else
{
System.Console.Error.WriteLine("non-empty string");
lines.Add(s);
System.Console.WriteLine("Got " + String.Join(" ", lines));
}
}
catch (Exception e)
{
System.Console.Error.WriteLine("Caught exception.");
}
}
}
}
Script:
./writer/bin/Debug/net6.0/writer.exe 3 | ./reader/bin/Debug/net6.0/reader.exe > o1
while :
do
./writer/bin/Debug/net6.0/writer.exe 3 | ./reader/bin/Debug/net6.0/reader.exe > oo
diff oo o1
if [ "$?" != "0" ]
then
break
fi
done
./writer/bin/Debug/net6.0/writer.exe 3 > in
while :
do
cat in | ./reader/bin/Debug/net6.0/reader.exe > oo
diff oo o1
if [ "$?" != "0" ]
then
break
fi
done
Output:
$ bash test.sh
Parse completed of C:\Users\kenne\Documents\AntlrVSIX\test\A.g4
reading...
non-empty string
reading...
Parse completed of C:\Users\kenne\Documents\AntlrVSIX\test\A.g4
read empty string
0a1,10
> Got {
> "FileName": "C:\\Users\\kenne\\Documents\\AntlrVSIX\\test\\A.g4"
> }
> {
> "FileName": "C:\\Users\\kenne\\Documents\\AntlrVSIX\\test\\A.g4"
> }
> {
> "FileName": "C:\\Users\\kenne\\Documents\\AntlrVSIX\\test\\A.g4"
> }
>
Parse completed of C:\Users\kenne\Documents\AntlrVSIX\test\A.g4
reading...
non-empty string
reading...
non-empty string
reading...
non-empty string
reading...
non-empty string
reading...
The problem is distinguishing between a non-blocking call in reader receiving an empty string vs. a true crash over in writer. The reader must be programmed to know when to exit the read loop.
- Write a newline in writer in case of crash. Reader will stop because it got something.
trxgrep:
public static void Main(string[] args)
{
try
{
new Program().MainInternal(args);
}
catch (Exception e)
{
System.Console.Error.WriteLine(e.ToString());
Environment.ExitCode = 1;
// Write something to avoid https://github.com/kaby76/Domemtech.Trash/issues/134
// and https://github.com/dotnet/runtime/issues/50780
System.Console.Out.WriteLine();
System.Console.Out.Flush();
System.Console.Out.Close();
}
}
trtext:
if (!(config.File != null && config.File != ""))
{
if (config.Verbose)
{
System.Console.Error.WriteLine("reading from stdin");
}
for (; ; )
{
lines = System.Console.In.ReadToEnd();
if (lines != null && lines != "")
{
break;
}
}
lines = lines.Trim();
}
else
{
if (config.Verbose)
{
System.Console.Error.WriteLine("reading from file >>>" + config.File + "<<<");
}
lines = File.ReadAllText(config.File);
}
Fixed in 0.18.0.