google / zx

A tool for writing better scripts

Home Page:https://google.github.io/zx/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Bug/Suggestion: configurable / overridable logging, default stderr

JohnnyMarnell opened this issue · comments

Expected vs Actual Behaviors, Examples

(Firstly, love this tool, much appreciated!)

I took a stab at a PR to address, as well, here:
#376

My expectation is that diagnostic logging / feedback messages should go to stderr like other CLI tools (e.g. curl, jq, ssh, tshark, etc), so that piping works as expected / stdout stream output can stay consistent even with verbose logging on.

e.g. with a zx script like (simple / trivial example) :

$.verbose = argv.logging
console.error('Pulling articles...')
const numArticles = 2, articles = []
for (let i = 1; i <= numArticles; i++) {
    const article = await $`curl ${argv.logging ? '-v' : ''} \
        https://jsonplaceholder.typicode.com/posts/${i}`
    articles.push(JSON.parse(article.stdout))
}
console.log(JSON.stringify({articles: articles}))

This zx script's conventional (stdout) output is JSON, thus piping to jq would work as expected:

zx ex.mjs | jq -c .

Pulling articles...
{"articles":[{"userId":1,"id":1,"title": ... }]}

However, this will fail, since zx currently logs diagnostics (like echo'd commands) to stdout, instead of more conventional stderr for this type of info:

zx ex.mjs --logging | jq -c .

Pulling articles...
* Connected to jsonplaceholder.typicode.com (172.67.131.170) port 443 (#0)
# etc, etc, plus much more verbose logging from curl... would work since it writes to stderr, 
#   but script fails with zx logging to stdout, since non json into piped stdout output:
parse error: Invalid numeric literal at line 1, column 2
node:events:505
      throw er; // Unhandled 'error' event
      ^

Whereas all of these work as expected, despite (sometimes very) verbose logging on (since they follow the POSIX standard of printing verbose diagnostic logging to stderr, not polluting expected output via stdout):

curl -v 'https://jsonplaceholder.typicode.com/posts/1' | jq .
jq -n '{ans: 42} | debug' | jq .
ssh -v "echo '{\"ans\":42}'" | jq .
# tshar, others, etc, etc.

Specifications

  • Version: N/A
  • Platform: N/A

Interesting. But main idea behind zx is to be bash script replacement.

In bash scripts subcommands print to stdout if not redirected. Same applies to zx.

Thanks for taking a look!

If this wasn't clear before, I'm referring to the need to change which stream solely for the diagnostic logging / verbose feedback messages case, to stderr instead of existing stdout, as seems standard to all CLI tooling (for diagnostic logging).

I agree any sub shell / tooling that a zx script calls, should inherit and honor which stream gets written to (in fact, in my examples, verbose logging like that of curl -v goes to stderr as it does were it called in a standard shell / non zx environment, and its standard / conventional output remains going to stdout, as expected).

The main problem is that zx is currently writing diagnostic logging output to stdout instead of stderr (an example is printing the underlying commands that are being executed when --verbose is on, which is quite helpful and nice for debugging(!) -- I want this useful feature to stay ofc, I just don't want it to break normal execution / piping when --verbose is on, as these other tools accomplish and the POSIX standard seems to indicate referring to "diagnostic" information going to stderr).

Since you mentioned bash specifically, here's a direct analogy to existing zx code:

bash -x -c 'echo foo' ;

And its output, written to streams:

+ echo foo
foo

The first line is going to stderr, since it's diagnostic info / feedback via -x, the second to stdout, which is the normal expected "output" stream for commands, including the simple echo of printing / echoing a string. To me, this is exactly analogous of a zx command (which, like any other shell command / tool, should be able to have output to be consumed by pipes / other machine processes, and the existing --verbose argument turns on verbose logging and diagnostics, particularly the exact same case here of printing which sub commands are being run, like -x for bash). And thus I think the standard is these messages should go to stderr.

Found some more related discussions / links on the web, if this helps explain the current problem:
POSIX standard:
https://pubs.opengroup.org/onlinepubs/9699919799/functions/stderr.html

At program start-up, three streams shall be predefined and need not be opened explicitly: standard input (for reading conventional input), standard output (for writing conventional output), and standard error (for writing diagnostic output).

https://julienharbulot.com/python-cli-streams.html

https://unix.stackexchange.com/questions/79315/when-to-use-redirection-to-stderr-in-shell-scripts

The easiest way to separate stderr from stdout : just imagine all your scripts output will be redirected to another command via pipe. In that case you should keep all notifications in stderr, as such unexpected information in stdout may break the pipe sequence.

https://stackoverflow.com/questions/4919093/should-i-log-messages-to-stderr-or-stdout
https://unix.stackexchange.com/questions/331611/do-progress-reports-logging-information-belong-on-stderr-or-stdout

I get it. Make sense. Let me think about it for a while.

https://github.com/google/zx/releases/tag/7.0.0

Now it's possible to configure it via $.log!

import { log } from 'zx'

$.log = (entry) => {
  if (entry.kind === 'sttdout') return
  log(entry)
}