macOS: TOML with Table values produces error on output

[Rezart Qelibari]

The Problem

Trying to output the data structure in TOML format (with -T option) produces an error on macOS High Sierra:

$ rq -tT 'id' < ./test.toml
 [ERROR] [rq] Encountered: values must be emitted before tables
 [ERROR] [rq] (Re-run with --trace or RUST_BACKTRACE=1 for a backtrace)

Investigations

Doing a web search with that error message led me to this source file of the toml crate project, that is responsible for serialising the data structure to a TOML file, that states:

Note that the TOML format has a restriction that if a table itself contains
tables, all keys with non-table values must be emitted first. This is
typically easy to ensure happens when you're defining a `struct` as you can
reorder the fields manually, but when working with maps (such as `BTreeMap`
or `HashMap`) this can lead to serialization errors. In those situations you
may use the `tables_last` function in this module [...]

And indeed, the error occurred only when there were tables to be serialised.

Solution

As stated in the serialisation source file, a solution is to make use of the tables_last function. I am currently figuring out, how I need to adapt the rq/src/value/toml.rs source file to do exactly that.

Any help is appreciated, especially as I have started learning Rust only a few days ago.

[Rezart Qelibari]

A workaround for now is to use the order preserving linked-hash-map instead of the BTreeMap.
Take a look at it on my fork.

[David Flemström]

Sounds like a reasonable change, maybe you want to make a PR? I don't think any logic should depend on object key order within rq right now (maybe some tests need tweaking)

[Nemo157]

Order preservation is not enough when you are doing something like --input-json --output-toml, the json may not be in the correct order to start with.