cli interface?
kvz opened this issue · comments
Hi, I just found this project via Hacker News, and wanted to thank you for releasing this!
We have the use case where we want to modify JSON values on disk in the fastest way possible (e.g. perhaps the json could be parsed in a streaming way, and as soon as the path has been found, forget about reading & decoding the rest, but make the change already and abort.
We're currently using elaborate sed regex matching and in-place replacement for this purpose which I'm obviously very ashamed of.
I was wondering if your tool could get a simple cli interface so we could benchmark it against what we currently use? It would be great if we could have something a bit more robust.
That is of course, if you think sjson might indeed be a suitable tool for this this use case?
All the best, Kevin
Yes sjson would work for this use case. The library does not parse irrelevant values. It will skip over values that do not apply and abort when the target value has been updated.
This could be ideal for a cli interface. Your suggestion prompted me to write one.
Hey @tidwall that is great news!
Does jsoned also support modifying the same file? Thinking about the case where I have a single file on disk that has the JSON, and I'm only interested in changing the first value of a key (i guess there can only ever be one : )
I guess I'm assuming that the infile would have to read the full file in order for the outfile to be written, and perhaps it would be possible to only change bytes in an existing file, and then abort reading it as soon as the replace succeeded?
Not sure if I'm making sense, or if it's even possible to write in parts of a file?
Yes, you can edit a file by assigning the input and output params to the same file. Though it does read the contents of the JSON into memory, modify the memory and write out to the file.
I'm not aware of a way to modify a json file in-place without rebuilding in memory first. I think it may be possible only if the new value requires fewer bytes than the previous value. For example, changing a value from "John" to "Andy" would work, but "John" to "Janet" would require a file rewrite. Perhaps it could use an intermediate temp file... something to think about.
That being said, jsoned is pretty darn quick. Should perform faster than sed, awk, or jq.
That sounds tremendous! We'll take it for a spin very soon!
Sounds good. I would love to hear your results.
Awesome @tidwall.
I just did some benchmarks with jsoned vs. regular sed for our use case. They are about the same in cpu usage, memory usage and time taken for a ~50kb json file with 50, 100, 200, 300, 400, 500 and 1000 subsequent changes to the file.
But for a 20MB big json file jsoned takes only half as long as sed does. :)
That's good news. Are you seeing cases where sed is faster than jsoned?
Correct me if I'm wrong @tim-kos but I thought in all cases, jsoned was faster, even if sometimes moderately.
Nope, that's right. For low concurrencies and low number of iterations they were pretty much the same. But for more complex operations jsoned was significantly faster.
Not sure if this is helpful or not for your use case, but I just added the new flag -O to jsoned which is optimized for value replacements. It's good for when you think that you might be doing an update/replace of value at a path.
For example:
echo '{"name":{"first":"Tom","last":"Smith"}}' | jsoned -v Tim -O name.first
The -O tells jsoned that the name.first likely exists so try a fasttrack operation first.
I'm closing this issue because there's now the https://github.com/tidwall/jj too. Feel free to reopen if needed. Thanks!
We’re very happy jj users now, thanks a lot!!
@kvz You're welcome :)