General issue with piping - all tests fail

Question

General issue with piping - all tests fail

lukele opened this issue a year ago · comments

Hi!

I was planning to rewrite a few of my shell scripts in python and stumbled across your project which looks really great.
Unfortunately I can't get any piping to work, neither on Linux nor on macOS.

Taken from your docs:

from sh import wc,ls

print(wc(ls("/home/<user>", "-l"), "-l"))

The expected result would be to get a number if I understand it correctly, since it should be the equivalent of
ls -l /home/user | wc -l

What I do get instead however is:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/<user>/.local/lib/python3.9/site-packages/sh.py", line 1523, in __call__
    rc = self.__class__.RunningCommandCls(cmd, call_args, stdin, stdout, stderr)
  File "/home/<user>/.local/lib/python3.9/site-packages/sh.py", line 749, in __init__
    self.wait()
  File "/home/<user>/.local/lib/python3.9/site-packages/sh.py", line 811, in wait
    self.handle_command_exit_code(exit_code)
  File "/home/<user>/.local/lib/python3.9/site-packages/sh.py", line 838, in handle_command_exit_code
    raise exc
sh.ErrorReturnCode_1: 

  RAN: /usr/bin/wc 'total 4
drwxr-xr-x 8 <user> <user> 4096 Jan  5 23:55 services
' -l

  STDOUT:


  STDERR:
/usr/bin/wc: 'total 4'$'\n''drwxr-xr-x 8 <user> <user> 4096 Jan  5 23:55 services'$'\n': No such file or directory

It appears as if sh passes the output that is supposed to be piped, as argument to the next command, instead of via pipe to stdin. I also tried variations of _piped and _iter without luck.

Any hint what I'm doing wrong would be great.

Lukas Pitschl · Answer 1 · Tue Feb 28 2023 03:43:06 GMT+0800 (China Standard Time)

After looking at the tests again, I believe I figured out that the docs are maybe no longer up to date and that the command to be piped to has to receive the input via _in

So the new syntax for the command would be:

print(wc("-l", _in=ls("/home/<user>", "-l")))

Is that how it is supposed to be working now?

Andrew · Answer 2 · Tue Feb 28 2023 06:21:29 GMT+0800 (China Standard Time)

Yes that is correct. There is an open issue for updating the docs to 2.0.0. If you would like to contribute to that effort, it is on the gh-pages branch

…

On Mon, Feb 27, 2023, 11:43 AM Lukas Pitschl ***@***.***> wrote: After looking at the tests again, I believe I figured out that the docs are maybe no longer up to date and that the command to be piped to has to receive the input via _in So the new syntax for the command would be: print(wc("-l", _in=ls("/home/<user>", "-l"))) Is that how it is supposed to be working now? — Reply to this email directly, view it on GitHub <#657 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAB7IKPSZHFAIGBIXGSOTADWZT7VNANCNFSM6AAAAAAVJ2DY3M> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

Lukas Pitschl · Answer 3 · Tue Feb 28 2023 07:06:43 GMT+0800 (China Standard Time)

I'll have to play around with it a bit to figure out more details. Could you share your reasoning behind that change, or can I find that in code?

Andrew · Answer 4 · Tue Feb 28 2023 08:39:17 GMT+0800 (China Standard Time)

I'm having a hard time finding the audit trail behind the decision. It could be that I never publicly shared my reasoning about it. I believe it caused confusion because the outer command would only assume that the first inner command was to be piped into stdin. Inner commands in other positions (other than the first) were not treated that way. Making it explicit about what is stdin and what are arguments removes that confusion. But not having the docs up to date adds that confusion back 🙃

Erik Cederstrand · Answer 5 · Tue Apr 18 2023 21:18:55 GMT+0800 (China Standard Time)

Closing since it's basically a duplicate of #656.