Jsondb ignore formatting characters upon loading

[Christoph Henrici]

First - off thanks for the great work! I have a feature suggestion: It would be nice, if jsondb can deal with whitespace resp. formatting characters upon loading.

Szeanario: To change data in the json files of jsondb i usually format the file in a Json Editor, which helps for editing the files. It seems that jsondb cannot deal with certain formatting. When i format a file for example with the JSON Editor Plugin in Eclipse on Windows i get upon loading the exception as seen in the attachted file exception.txt
with the file patch.txt ( I changed the suffix from json to txt because it's seem Github does'nt know json files)

Json db actually then deletes all data in file and starts with a fresh file.

I think the future to able to edit "live" and then reload the file is very good.

[Farooq Khan]

@chhex Thanks for using JsonDB and taking the time to report this issue.

Yes, this is a problem. Its easier to deal with the files if one line of data is treated as just one row/document. but as you highlighted editing these .json files is not really user-friendly. And I have often thought of ways to fix this, but never could determine the best way. A few ways I think this can be fixed:

• Accept a setting for pretty-read=true and then expect the files to be formatted for readability then parse for } which is not followed by a , and break the rows at that point.
• Additionally accept a setting for pretty-print=true and then when saving the files format them accordingly
• OR accept a setting lines-per-row and then parse the file reading lines-per-row each time

Could you maybe suggest more ways to do this or your view on this.

Could other users of JsonDB maybe suggest ways to fix this. I will be happy to fix this if I have some feedback.

[Christoph Henrici]

@FarooqKhan best thanks for your time and consideration.

I agree it's not a trivial problem. I think your suggestions are helpful.
How about only the pretty-print=true option and then jsondb can make assumptions that the formatting is left unchanged? At least for a start.
That said: i have to try it out to give you definite feedback. I would be happy to try it out.

[Deleted user]

@FarooqKhan @chhex
What if the internal representation of the collection is a bit different?
Instead of having:

{"schemaVersion":"1.0"}
{"id":"x", "field":"y"}
{"id":"a", "field":"b"}

have something like

{
  "schemaVersion": "1.0",
   "items" : [
    {"id":"x", "field":"y"},
    {"id":"a", "field":"b"}
  ]
}

Since the latter json is in correct form, it does not matter whether you pretty-print it or not.
Yes, it does add two levels of nesting, but on the other hand using any json parser is possible.
Also, this form allows you to insert as much metadata (like storing indexes) as you want and easily parse it back later.

PS: I can see that having an item per line somewhat speeds up the reading-writing-parsing from the file.

[Farooq Khan]

@chhex
I created a Atom Editor plugin to solve this problem that you are facing jsondb-atom-plugin.

The plugin is work in progress but should be usable as it is now.

If you get some time try using it, I would appreciate any feedback you have about the plugin.

I know its a special way of editing these files but I think this is much simpler way to solve this problem then making Jsondb itself handle the formatting, I suspect it will make Jsondb slow and code will take time to perfect