java-jwt-benchmark
Project for benchmarking popular Json Web Token (JWT) frameworks for Java using JMH.
Supported frameworks:
- java-jwt from Auth0
- jjwt from jwtk
- Okta JWT Verifier for Java from Okta
- fusionauth-jwt from FusionAuth
- Nimbus-JOSE-JWT from connect2id
License
Obtain
The project is based on Gradle.
Usage
Modify the build version to your current snapshot, then run
./gradlew --stop && ./gradlew clean jmhClasses jmh --refresh-dependencies --info
Note: Requires Java 8 to build, there is some classpath issue with Java 11 which I've yet to understand.
Disable CPU Boost
Temporarily disable boost (untill next reboot) for AMD:
echo "0" | sudo tee /sys/devices/system/cpu/cpufreq/boost
The JMH plugin seems to have trouble refreshing the project, so restart the Gradle deamon before running.
Benchmarks
- Parse token (if possible)
- Verify token
- Verify token, get claim.
General performance observations
There is not much headroom for improving the performance of RSA-based signatures; most (> 90%) of the CPU time goes to signature verification.
This observation is based on two simple 'baseline' RSA-based JWT verifiers included in this project. They perform about 5% faster than the fastest JWT libraries, but then only does signature verification and no JSON parsing / field validation. So it seems likely this is the upper limit.
Latest results
Running for OpenJDK 11 build 11.0.18+10 and OpenJDK 17 build 17.0.6+10, on Fedora Linux 6.1.8-200, AMD Ryzen 5950.
Result visualizations:
Summary: For JWT verification fusionauth-jwt and java-jwt are tied for first place. fusionauth-jwt can parse faster (without verification).
Previous results
Note that the relative number matters, benchmarks are not necessarily run on the same machine, operating system or Java version.
Version 1.4.0:
Running for OpenJDK 11 build 11.0.11.0.9, Fedora Linux 5.13.4-200, AMD Ryzen 7 PRO 3700U laptop.
A visualization_v1.4.0 is available. In short, fusionauth-jwt is the fastest parser, but Java-jwt is very close.
Version 1.3.0:
Running for OpenJDK 1.8.0 build 275-b01, Fedora Linux 5.10.7-200, AMD 5950x.
A visualization_v1.3.0 is available, summery:
Framework | Version | Verify (op/s) | Claim (op/s) | Parse (op/s) |
---|---|---|---|---|
java-jwt | 3.12.0/0.15.0 | 30.1k | 29.9k | 387k |
fusionauth-jwt | 4.0.1 | 32.6k | 32.4k | 1196k |
jjwt | 0.9.1 | 31.4k | 30.7k | |
Nimbus-JOSE-JWT | 9.4.1 | 19.7k | 18.9k | 945k |
Okta JWT Verifier for Java | 0.5.0 | 1.3k | 1.3k |
In short, fusionauth-jwt is the fastest parser. Parsing without validation is now faster than before for java-jwt, otherwise no improvements.
Version 1.2.0:
Running for OpenJDK 1.8.0 build 232, Linux 5.3.7.
A visualization_v1.2.0 is available, summery:
Framework | Version | Verify (op/s) | Claim (op/s) | Parse (op/s) |
---|---|---|---|---|
java-jwt | 3.8.3/0.9.0 | 14.6k | 14.6k | 230.8k |
jjwt | 0.9.1 | 14.9k | 14.5k | |
Okta JWT Verifier for Java | 0.4.0 | 0.6k | 0.6k | |
fusionauth-jwt | 3.1.6 | 15.9k | 15.7k | 581.5k |
In short, fusionauth-jwt is the new fastest parser. Parsing without validation is now faster than before for java-jwt, otherwise no improvements.
Version 1.1.0:
Running for OpenJDK 1.8.0 build 191.
A visualization_v1.1.0 is available, summery:
Framework | Version | Verify (op/s) | Claim (op/s) |
---|---|---|---|
java-jwt | 3.7.0/0.7.0 | 14.5k | 14.6k |
jjwt | 0.9.1 | 14.7k | 14.9k |
Okta JWT Verifier for Java | 0.4.0 | 0.61k | 0.61k |
In short, java-jwt did catch up with jjwt, they were essentially just as fast. The Okta JWT verifiser regressed into a total disaster.
Only java-jwt seems to parse without validation, at about 52.3k operations per second.
Version 1.0.0:
Running for Oracle JDK 1.8 build unknown-
A visualization_v1.0.0 is available, summery:
Framework | Version | Verify (op/s) | Claim (op/s) |
---|---|---|---|
java-jwt | 3.3.0/0.4.0 | 14.2k | 13.9k |
jjwt | 0.9.0 | 19k | 18.3k |
Okta JWT Verifier for Java | 0.3.0 | 17.1k | 17.1k |
Only java-jwt seems to parse without validation, at about 65.2k operations per second.
History
- 1.x: Added baseline JWT verifiers
- 1.5.0: Updated dependencies
- 1.3.0: Updated dependencies, added Nimbus JOSE + JWT
- 1.2.0: Added FusionAuth
- 1.1.0: Bumped versions after accepted performance-enhancing PR#255 for java-jwt.
- 1.0.0: Initial version