json2yaml line-wraps the output

[Colin Viebrock]

Part of my JSON file has the following:

{
  "balanceHeld": {
    "description": "Balance of the account held for transferring to other users (in cents)",
    "type": "integer"
  },
}

json2yaml converts this to:

balanceHeld:
  description: Balance of the account held for transferring to other users (in
    cents)
  type: integer

It looks like all long lines are being wrapped at 80 characters or so. But that line break that's being added in the description property is causing another tool to fail importing, because I don't think wrapping long lines like that results in a valid YAML file.

Could you add a flag to prevent line wrapping? Or output long lines with the "pipe" notation:

balanceHeld:
  description: |
    Balance of the account held for transferring to other users (in
    cents)
  type: integer

[D. Bohdan]

I cannot reproduce this with Go 1.4.2 and the latest versions of the dependencies on Linux x86_64. Could you run this script in the repository with a built binary and tell me what it outputs for you?

#!/bin/sh
set -e
cat >wrap.json <<END
{
  "balanceHeld": {
    "description": "Balance of the account held for transferring to other users (in cents)",
    "type": "integer"
  }
}
END
cat >wrap.reference.yaml <<END
balanceHeld:
  description: Balance of the account held for transferring to other users (in cents)
  type: integer

END
./remarshal -if=json -of=yaml -i=wrap.json > wrap.yaml
diff wrap.reference.yaml wrap.yaml && echo same
cp remarshal json2yaml
./json2yaml wrap.json > wrap.yaml
diff wrap.reference.yaml wrap.yaml && echo same
rm json2yaml

[Colin Viebrock]

I will first admit that I installed the binaries on OSX via brew install remarshal. Using those prebuilt binaries, the above works.

However, add a bunch more text to the description field in your example, and it does the line wrapping:

#!/bin/sh
set -e
cat >wrap.json <<END
{
  "balanceHeld": {
    "description": "Balance of the account held for transferring to other users (in cents) and then a bunch more text after that",
    "type": "integer"
  }
}
END
cat >wrap.reference.yaml <<END
balanceHeld:
  description: Balance of the account held for transferring to other users (in cents) and then a bunch more text after that
  type: integer

END
./remarshal -if=json -of=yaml -i=wrap.json > wrap.yaml
diff wrap.reference.yaml wrap.yaml && echo same
cp remarshal json2yaml
./json2yaml wrap.json > wrap.yaml
diff wrap.reference.yaml wrap.yaml && echo same
rm json2yaml

The output from both diffs:

2c2,3
<   description: Balance of the account held for transferring to other users (in cents) and then a bunch more text after that
---
>   description: Balance of the account held for transferring to other users (in cents)
>     and then a bunch more text after that

[D. Bohdan]

Right, I've reproduced it now. This appears to be an upstream bug: go-yaml/yaml#166.

While I could fix the bug upstream, there is another problem with go-yaml: #2. Neither go-yaml nor BurntSushi/toml seems to have a more a actively maintained alternative that handles the general case of serializing arbitrarily nested map/array data structures via reflection, so rewriting remarshal in another language that has such alternatives may be a better long-term investment. I'll look into it when I have the time.

[Colin Viebrock]

Thanks for looking into this!

FWIW, in the interest of getting work done today, I found an node package that does what I needed: https://www.npmjs.com/package/json2yaml

[D. Bohdan]

Ah, good. This alternative should be useful to whoever else runs into this bug until it is fixed.

JavaScript sure does have well-maintained libraries for YAML and TOML. One strategy I'm considering for a rewrite is to package them in a static binary with Duktape, an embedded JS interpreter. This should be as convenient as the binaries built with Go.

[D. Bohdan]

Implemented in v0.5. Try

./remarshal.py -if=json -of=yaml '--yaml-style=|' wrap.json