skjolber / datasets-json-benchmark

Json parser benchmarks for specific datasets

datasets-json-benchmark

Project for benchmarking popular JSON parsers for Java using JMH on specific datasets.

Supported frameworks:

• Jackson
• Gson
• Jsoniter

Supported datasets:

• NVE database
  • Full schema parsing
  • Reduced schema parsing (skipping values not actually in use)
    • As in fields accessed in OWASP Dependency Check
  • Ecosystem classification
    • As in OWASP Dependency Check

License

Apache 2.0

Obtain

The project is based on Gradle.

Usage

Modify the build version to your current snapshot, then run

./gradlew --stop && ./gradlew clean jmhClasses jmh --refresh-dependencies --info

The JMH plugin seems to have trouble refreshing the project, so restart the Gradle deamon before running.

Benchmarks

• Deserialize to objects

Results

Java 8

Running for OpenJDK 1.8.0 build 232, Linux 5.3.14.

A visualization_v1.1.0-JDK8 is available, summery:

JSON Parsing:

• GSON is slowest
• Jackson is 15-25% faster
• Joniter is at 30-40% faster than Jackson
• Reduced schema parsing is faster.

In short, jsoniter pregenerated parser is the fastest parser. However a bug was detected in the stream handling, so the library seems not to have sufficient quality testing / code coverage.

Ecosystem search

The so-called Aho Corasick algorithm is more than 20x faster than the classic String.containsIgnoreCase(..).

Java 11

Running for OpenJDK 11 build 11.0.5+10-LTS, Linux 5.3.14.

A visualization_v1.1.0-JDK11 is available; same results as above.

• Jackson does relatively better
• Performance regressions for the other two

History

• 1.1.0: Added ecosystem search. Fixed a bug in the Jackson benchmark.
• 1.0.0: Initial version