Newline in xlsx header causes crash

Question

Newline in xlsx header causes crash

infojunkie opened this issue 2 years ago · comments

Using the attached xlsx sheet, running dsq newline.xlsx --schema generates the following error:

json: error calling MarshalJSON for type *runner.Shape: invalid character '\n' in string literal

Running dsq newline.xlsx does not crash:

[{"Header with \nNewline":"1"},
{"Header with \nNewline":"2"},
{"Header with \nNewline":"3"},
{"Header with \nNewline":"4"}]

Removing the newline from the header cell no longer crashes the schema command.

Phil Eaton · Answer 1 · Sun Jun 26 2022 22:35:51 GMT+0800 (China Standard Time)

This may also be related to #79 and https://github.com/multiprocessio/dsq/releases/tag/0.20.2

Phil Eaton · Answer 2 · Mon Jun 27 2022 02:45:56 GMT+0800 (China Standard Time)

Tested this out and it's its own issue. I think I'll deal with this be dropping all newlines in column names for all file types since they all seem to break SQLite.

Phil Eaton · Answer 3 · Mon Jun 27 2022 03:03:22 GMT+0800 (China Standard Time)

With the help of Twitter I figured out a workaround :D

dsq newline.xlsx "$(printf "select [Header with \nNewline] from {}")"

Phil Eaton · Answer 4 · Mon Jun 27 2022 03:03:57 GMT+0800 (China Standard Time)

The issue is that bash won't insert a literal newline but other tools like printf can do that.

Karim Ratib · Answer 5 · Mon Jun 27 2022 06:34:34 GMT+0800 (China Standard Time)

Thanks for the workaround! Yes, dropping the newline would simplify handling.