JSON Parse Practical Example
April 18, 2017 · View on GitHub
| Library | Mean in ms (lower is better) 0.2.7 | 0.2.8 | 0.3.6 |
|---|---|---|---|
| pjson | 10.183120 ms | 9.526570 ms | 9.382459 ms |
| clj-json | 41.713927 ms | 42.148021 ms | |
| boon | 44.195550 ms | 45.949513 ms | 40.701913 ms |
| cheshire | 49.784366 ms | 50.600446 ms | |
| data.json | 165.176958 ms | 171.720518 ms |
Goal: JSON Parse Practical Example
-----
Case: :data.json
Evaluation count : 360 in 60 samples of 6 calls.
Execution time mean : 171.720518 ms
Execution time std-deviation : 3.128919 ms
Execution time lower quantile : 167.051332 ms ( 2.5%)
Execution time upper quantile : 176.926498 ms (97.5%)
Overhead used : 1.881626 ns
Found 1 outliers in 60 samples (1.6667 %)
low-severe 1 (1.6667 %)
Variance from outliers : 7.7886 % Variance is slightly inflated by outliers
Case: :pjson
Evaluation count : 6360 in 60 samples of 106 calls.
Execution time mean : 9.382459 ms
Execution time std-deviation : 96.696634 µs
Execution time lower quantile : 9.258054 ms ( 2.5%)
Execution time upper quantile : 9.694433 ms (97.5%)
Overhead used : 1.553512 ns
Found 4 outliers in 60 samples (6.6667 %)
low-severe 1 (1.6667 %)
low-mild 3 (5.0000 %)
Variance from outliers : 1.6389 % Variance is slightly inflated by outliers
Case: :cheshire
Evaluation count : 1200 in 60 samples of 20 calls.
Execution time mean : 50.600446 ms
Execution time std-deviation : 1.236219 ms
Execution time lower quantile : 49.122398 ms ( 2.5%)
Execution time upper quantile : 54.349364 ms (97.5%)
Overhead used : 1.881626 ns
Found 3 outliers in 60 samples (5.0000 %)
low-severe 2 (3.3333 %)
low-mild 1 (1.6667 %)
Variance from outliers : 12.5577 % Variance is moderately inflated by outliers
Case: :boon
Evaluation count : 1500 in 60 samples of 25 calls.
Execution time mean : 40.701913 ms
Execution time std-deviation : 587.048788 µs
Execution time lower quantile : 39.874727 ms ( 2.5%)
Execution time upper quantile : 42.248703 ms (97.5%)
Overhead used : 1.553512 ns
Found 3 outliers in 60 samples (5.0000 %)
low-severe 3 (5.0000 %)
Variance from outliers : 1.6389 % Variance is slightly inflated by outliers
Case: :clj-json
Evaluation count : 1500 in 60 samples of 25 calls.
Execution time mean : 42.148021 ms
Execution time std-deviation : 2.859100 ms
Execution time lower quantile : 40.430278 ms ( 2.5%)
Execution time upper quantile : 49.266426 ms (97.5%)
Overhead used : 1.881626 ns
Found 5 outliers in 60 samples (8.3333 %)
low-severe 3 (5.0000 %)
low-mild 2 (3.3333 %)
Variance from outliers : 51.7236 % Variance is severely inflated by outliers
JSON Parse Benchmark
| Library | Mean in ms (lower is better) 0.2.7 | 0.2.8 | 0.3.6 |
|---|---|---|---|
| pjson | 3.471202 ms | 3.397253 ms | 3.239881 ms |
| clj-json | 30.000884 ms | 30.696739 ms | |
| boon | 7.775955 ms | 8.178839 ms | 6.669186 ms |
| cheshire | 26.599338 ms | 27.984381 ms | |
| data.json | 74.359619 ms | 76.775012 ms |
Goal: JSON Parse Benchmark
-----
Case: :clj-json
Evaluation count : 1980 in 60 samples of 33 calls.
Execution time mean : 30.696739 ms
Execution time std-deviation : 546.312660 µs
Execution time lower quantile : 29.917998 ms ( 2.5%)
Execution time upper quantile : 32.038518 ms (97.5%)
Overhead used : 1.881626 ns
Found 4 outliers in 60 samples (6.6667 %)
low-severe 4 (6.6667 %)
Variance from outliers : 7.7624 % Variance is slightly inflated by outliers
Case: :boon
Evaluation count : 9000 in 60 samples of 150 calls.
Execution time mean : 6.669186 ms
Execution time std-deviation : 146.054908 µs
Execution time lower quantile : 6.563548 ms ( 2.5%)
Execution time upper quantile : 7.100086 ms (97.5%)
Overhead used : 1.553512 ns
Found 5 outliers in 60 samples (8.3333 %)
low-severe 1 (1.6667 %)
low-mild 4 (6.6667 %)
Variance from outliers : 9.4570 % Variance is slightly inflated by outliers
Case: :pjson
Evaluation count : 18780 in 60 samples of 313 calls.
Execution time mean : 3.239881 ms
Execution time std-deviation : 35.395760 µs
Execution time lower quantile : 3.206980 ms ( 2.5%)
Execution time upper quantile : 3.329477 ms (97.5%)
Overhead used : 1.553512 ns
Found 5 outliers in 60 samples (8.3333 %)
low-severe 3 (5.0000 %)
low-mild 2 (3.3333 %)
Variance from outliers : 1.6389 % Variance is slightly inflated by outliers
Case: :cheshire
Evaluation count : 2160 in 60 samples of 36 calls.
Execution time mean : 27.984381 ms
Execution time std-deviation : 695.037351 µs
Execution time lower quantile : 27.106720 ms ( 2.5%)
Execution time upper quantile : 29.370169 ms (97.5%)
Overhead used : 1.881626 ns
Found 5 outliers in 60 samples (8.3333 %)
low-severe 4 (6.6667 %)
low-mild 1 (1.6667 %)
Variance from outliers : 12.5828 % Variance is moderately inflated by outliers
Case: :data.json
Evaluation count : 840 in 60 samples of 14 calls.
Execution time mean : 76.775012 ms
Execution time std-deviation : 1.458656 ms
Execution time lower quantile : 75.032641 ms ( 2.5%)
Execution time upper quantile : 80.653407 ms (97.5%)
Overhead used : 1.881626 ns
Found 5 outliers in 60 samples (8.3333 %)
low-severe 3 (5.0000 %)
low-mild 2 (3.3333 %)
Variance from outliers : 7.8323 % Variance is slightly inflated by outliers
JSON Parse Benchmark - parse all fields and substructures
| Library | Mean in ms (lower is better) 0.2.7 | 0.2.8 | 0.3.6 |
|---|---|---|---|
| pjson | 210.109698 ms | 214.128818 ms | 154.815778 ms |
| clj-json | 234.800428 ms | 244.911982 ms | 185.095822 ms |
| boon | 226.109138 ms | 235.744988 ms | 167.414613 ms |
| cheshire | 233.513268 ms | 242.169108 ms | 180.337285 ms |
| data.json | 282.268952 ms | 292.574602 ms | 221.465102 ms |
Goal: JSON Parse Benchmark - parse all fields and substructures
-----
Case: :data.json
Evaluation count : 300 in 60 samples of 5 calls.
Execution time mean : 221.465102 ms
Execution time std-deviation : 2.685839 ms
Execution time lower quantile : 217.661820 ms ( 2.5%)
Execution time upper quantile : 227.296331 ms (97.5%)
Overhead used : 1.553512 ns
Found 4 outliers in 60 samples (6.6667 %)
low-severe 3 (5.0000 %)
low-mild 1 (1.6667 %)
Variance from outliers : 1.6389 % Variance is slightly inflated by outliers
Case: :boon
Evaluation count : 420 in 60 samples of 7 calls.
Execution time mean : 167.414613 ms
Execution time std-deviation : 1.828060 ms
Execution time lower quantile : 165.567706 ms ( 2.5%)
Execution time upper quantile : 172.131859 ms (97.5%)
Overhead used : 1.553512 ns
Found 4 outliers in 60 samples (6.6667 %)
low-severe 3 (5.0000 %)
low-mild 1 (1.6667 %)
Variance from outliers : 1.6389 % Variance is slightly inflated by outliers
Case: :cheshire
Evaluation count : 360 in 60 samples of 6 calls.
Execution time mean : 180.337285 ms
Execution time std-deviation : 3.151541 ms
Execution time lower quantile : 176.927905 ms ( 2.5%)
Execution time upper quantile : 187.636870 ms (97.5%)
Overhead used : 1.553512 ns
Found 8 outliers in 60 samples (13.3333 %)
low-severe 7 (11.6667 %)
low-mild 1 (1.6667 %)
Variance from outliers : 6.2877 % Variance is slightly inflated by outliers
Case: :pjson
Evaluation count : 420 in 60 samples of 7 calls.
Execution time mean : 154.815778 ms
Execution time std-deviation : 1.529529 ms
Execution time lower quantile : 153.082568 ms ( 2.5%)
Execution time upper quantile : 157.792013 ms (97.5%)
Overhead used : 1.553512 ns
Found 3 outliers in 60 samples (5.0000 %)
low-severe 2 (3.3333 %)
low-mild 1 (1.6667 %)
Variance from outliers : 1.6389 % Variance is slightly inflated by outliers
Case: :clj-json
Evaluation count : 360 in 60 samples of 6 calls.
Execution time mean : 185.095822 ms
Execution time std-deviation : 2.106442 ms
Execution time lower quantile : 182.398961 ms ( 2.5%)
Execution time upper quantile : 190.273053 ms (97.5%)
Overhead used : 1.553512 ns
Found 2 outliers in 60 samples (3.3333 %)
low-severe 2 (3.3333 %)
Variance from outliers : 1.6389 % Variance is slightly inflated by outliers
JSON Encode Map
This benchmarks aims to test encoding speed from standard clojure/java data structures.
The test message is parsed with clj-json which does this without any lazy intermediates.
| Library | Mean in ms (lower is better) 0.2.7 | 0.2.8 | 0.3.6 |
|---|---|---|---|
| pjson | 9.881341 ms | 10.291857 ms | 9.577133 ms |
| clj-json | 9.240113 ms | 8.785915 ms | 7.440156 ms |
| boon | 60.509740 ms | 65.338146 ms | 56.358606 ms |
| cheshire | 19.348805 ms | 20.054495 ms | 21.417734 ms |
| data.json | 76.980829 ms | 79.525906 ms | 73.388868 ms |
Goal: JSON Encode Map Benchmark
-----
Case: :boon
Evaluation count : 1080 in 60 samples of 18 calls.
Execution time mean : 56.358606 ms
Execution time std-deviation : 1.507475 ms
Execution time lower quantile : 55.202649 ms ( 2.5%)
Execution time upper quantile : 59.223079 ms (97.5%)
Overhead used : 1.553512 ns
Found 4 outliers in 60 samples (6.6667 %)
low-severe 1 (1.6667 %)
low-mild 3 (5.0000 %)
Variance from outliers : 14.1810 % Variance is moderately inflated by outliers
Case: :pjson
Evaluation count : 6300 in 60 samples of 105 calls.
Execution time mean : 9.577133 ms
Execution time std-deviation : 65.927621 µs
Execution time lower quantile : 9.473455 ms ( 2.5%)
Execution time upper quantile : 9.736950 ms (97.5%)
Overhead used : 1.553512 ns
Found 3 outliers in 60 samples (5.0000 %)
low-severe 1 (1.6667 %)
low-mild 2 (3.3333 %)
Variance from outliers : 1.6389 % Variance is slightly inflated by outliers
Case: :data.json
Evaluation count : 840 in 60 samples of 14 calls.
Execution time mean : 73.388868 ms
Execution time std-deviation : 416.120409 µs
Execution time lower quantile : 72.807563 ms ( 2.5%)
Execution time upper quantile : 74.423537 ms (97.5%)
Overhead used : 1.553512 ns
Found 2 outliers in 60 samples (3.3333 %)
low-severe 2 (3.3333 %)
Variance from outliers : 1.6389 % Variance is slightly inflated by outliers
Case: :cheshire
Evaluation count : 2820 in 60 samples of 47 calls.
Execution time mean : 21.417734 ms
Execution time std-deviation : 186.582387 µs
Execution time lower quantile : 21.146894 ms ( 2.5%)
Execution time upper quantile : 21.862683 ms (97.5%)
Overhead used : 1.553512 ns
Found 3 outliers in 60 samples (5.0000 %)
low-severe 3 (5.0000 %)
Variance from outliers : 1.6389 % Variance is slightly inflated by outliers
Case: :clj-json
Evaluation count : 8160 in 60 samples of 136 calls.
Execution time mean : 7.440156 ms
Execution time std-deviation : 44.436823 µs
Execution time lower quantile : 7.360257 ms ( 2.5%)
Execution time upper quantile : 7.527837 ms (97.5%)
Overhead used : 1.553512 ns
Found 2 outliers in 60 samples (3.3333 %)
low-severe 2 (3.3333 %)
Variance from outliers : 1.6389 % Variance is slightly inflated by outliers
JSON Encode Simple
This benchmarks reads the message using each API's decode/parse functions and then without edits
encodes the message back to json.
Note: pjson can remember the original json message for sub messages, such that if no edits were made the messages' original json string is written back.
Summary of the benchmark results are below (in order of faster to slowest).
| Library | Mean in ms (lower is better) 0.2.7 | 0.2.8 | 0.3.6 |
|---|---|---|---|
| pjson | 2.380149 ms | 2.394470 ms | 2.431828 ms |
| clj-json | 9.208672 ms | 9.950834 ms | 8.026480 ms |
| boon | 10.196466 ms | 11.121667 ms | 9.488874 ms |
| cheshire | 19.375317 ms | 20.302390 ms | 21.962842 ms |
| data.json | 74.689144 ms | 82.368058 ms | 78.556321 ms |
Goal: JSON Encode Simple
-----
Case: :clj-json
Evaluation count : 7800 in 60 samples of 130 calls.
Execution time mean : 8.026480 ms
Execution time std-deviation : 305.671832 µs
Execution time lower quantile : 7.589653 ms ( 2.5%)
Execution time upper quantile : 8.600770 ms (97.5%)
Overhead used : 1.553512 ns
Case: :pjson
Evaluation count : 25080 in 60 samples of 418 calls.
Execution time mean : 2.431828 ms
Execution time std-deviation : 42.347751 µs
Execution time lower quantile : 2.352209 ms ( 2.5%)
Execution time upper quantile : 2.506976 ms (97.5%)
Overhead used : 1.553512 ns
Case: :data.json
Evaluation count : 840 in 60 samples of 14 calls.
Execution time mean : 78.556321 ms
Execution time std-deviation : 2.979267 ms
Execution time lower quantile : 73.135290 ms ( 2.5%)
Execution time upper quantile : 83.312295 ms (97.5%)
Overhead used : 1.553512 ns
Case: :cheshire
Evaluation count : 2880 in 60 samples of 48 calls.
Execution time mean : 21.962842 ms
Execution time std-deviation : 756.145386 µs
Execution time lower quantile : 21.024825 ms ( 2.5%)
Execution time upper quantile : 23.578767 ms (97.5%)
Overhead used : 1.553512 ns
Case: :boon
Evaluation count : 6360 in 60 samples of 106 calls.
Execution time mean : 9.488874 ms
Execution time std-deviation : 419.549008 µs
Execution time lower quantile : 8.984589 ms ( 2.5%)
Execution time upper quantile : 10.278705 ms (97.5%)
Overhead used : 1.553512 ns
Number parse Benchmark
This benchmark is used to show the difference between the manual string to integer reading in pjson and the java Long.valueOf
| Library | Mean in ms (lower is better) 0.2.8 |
|---|---|
| pjson number-util-parse5 | 12.869024 ns |
| java Long.valueOf | 38.067585 ns |