KrakenSerializer.md

May 6, 2018 ยท View on GitHub

Serialization / DesertOctopus.KrakenSerializer

The main use of this serializer is to serialize DTOs using binary serialization. The serialization engine is implemented using expression trees which mean a small performance hit the first type you serialize a type for the creation of the expression tree. Subsequent serialization for the same type can use the compiled expression tree without having to suffer the performance hit.

The main pros of DesertOctopus.KrakenSerializer are:

  • Binary serialization for use in remote caching server scenarios
  • Does not require objects to be decorated with the Serializable attribute
    • It is up to the user of the serializer to ensure that the objects can be safely serialized
  • Serialize all fields (private, public, etc) of an object
  • Thread safe
  • Supports interface ISerializable
    • Your class needs to have the corresponding serialization constructor otherwise it will be serialized as a normal class
  • Respect the NotSerialized attribute
  • Supported types:
    • All primitive types
    • Multi dimensional arrays
    • Jagged arrays
    • Class / Struct
    • ExpandoObject
    • Basic support for IEnumerable<> and IQueryable<>
      • Don't go crazy and try to serialize a GroupedEnumerable or something similar
  • No need to know the type of the object to deserialize. The object's type is embedded in the serialized payload.
  • Automatic abort if an object's definition (e.g.: number of fields, name of a field, etc) was modified.
  • Can handle circular references
    • There is one case that is not supported: a dictionary with a reference to itself
  • Good unit tests
  • You can say "Release the Kraken" when you serialize something

The mains cons are:

  • It's not the fastest serializer around. Supporting the above features comes at the price of speed/cpu usage.
  • Including the objects' types increase the size of the payload however special attention has been made to minimize the size of serialized object.

Usage


// serialize an object
var objectToSerialize = new MyDto();
var serializedBytes = KrakenSerializer.Serialize(objectToSerialize);

// deserialize the object
var deserializedObject = KrakenSerializer.Deserialize<MyDto>(serializedBytes);

Benchmark

This benchmark serialize and deserialize a fairly large object containing array, lists and dictionaries.


BenchmarkDotNet=v0.10.14, OS=Windows 8.1 (6.3.9600.0)
Intel Core i5-4690 CPU 3.50GHz (Haswell), 1 CPU, 4 logical and 4 physical cores
Frequency=14318180 Hz, Resolution=69.8413 ns, Timer=HPET
  [Host]     : .NET Framework 4.7 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.2563.0
  DefaultJob : .NET Framework 4.7 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.2563.0
MethodMeanErrorStdDevSize (bytes)Gen 0Gen 1Gen 2Allocated
JsonSerialization672.6 us18.7013 us19.2048 us37375146.48440.9766-454.55 KB
JsonDeserialization673.5 us5.5046 us5.1490 us36.13289.7656-120.06 KB
KrakenSerialization314.8 us4.8563 us4.5426 us1471930.2734--93.17 KB
KrakenDeserialization155.7 us1.0993 us1.0283 us21.7285--67.03 KB
KrakenSerializationWithOmittedRootType310.7 us3.4887 us3.0926 us1460329.7852--93 KB
KrakenDeserializationWithOmittedRootType155.8 us1.2038 us1.1260 us21.7285--66.9 KB
BinaryFormatterSerialization996.6 us4.7385 us4.2006 us72220164.062542.968841.0156636.22 KB
BinaryFormatterDeserialization855.7 us16.1918 us16.6278 us89.843830.2734-390.5 KB
NetSerializerSerialization143.5 us0.5811 us0.5436 us2459929.7852--92.03 KB
NetSerializerDeserialization143.2 us2.1457 us2.0071 us32.22660.7324-99.58 KB

This benchmark serialize and deserialize a normal sized object that contains all primitives types.


BenchmarkDotNet=v0.10.14, OS=Windows 8.1 (6.3.9600.0)
Intel Core i5-4690 CPU 3.50GHz (Haswell), 1 CPU, 4 logical and 4 physical cores
Frequency=14318180 Hz, Resolution=69.8413 ns, Timer=HPET
  [Host]     : .NET Framework 4.7 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.2563.0
  DefaultJob : .NET Framework 4.7 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.2563.0
MethodMeanErrorStdDevSize (bytes)Gen 0Allocated
JsonSerialization15,147.9 ns92.596 ns82.084 ns5913.11289826 B
JsonDeserialization15,180.5 ns168.176 ns149.084 ns1.15973720 B
KrakenSerialization2,451.6 ns26.867 ns23.817 ns2310.67142120 B
KrakenDeserialization1,434.2 ns10.059 ns9.409 ns0.43681376 B
KrakenSerializationWithOmittedRootType1,758.4 ns19.239 ns17.996 ns1200.56651784 B
KrakenDeserializationWithOmittedRootType1,130.4 ns10.443 ns9.768 ns0.2880912 B
BinaryFormatterSerialization39,372.8 ns345.972 ns323.623 ns17357.812524648 B
BinaryFormatterDeserialization22,631.6 ns106.645 ns99.756 ns4.791315152 B
NetSerializerSerialization770.1 ns4.145 ns3.878 ns1050.1898600 B
NetSerializerDeserialization637.6 ns8.682 ns8.121 ns0.1240392 B

This benchmark serialize and deserialize an array of 100000 ints.


BenchmarkDotNet=v0.10.14, OS=Windows 8.1 (6.3.9600.0)
Intel Core i5-4690 CPU 3.50GHz (Haswell), 1 CPU, 4 logical and 4 physical cores
Frequency=14318180 Hz, Resolution=69.8413 ns, Timer=HPET
  [Host]     : .NET Framework 4.7 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.2563.0
  DefaultJob : .NET Framework 4.7 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.2563.0
MethodMeanErrorStdDevSize (bytes)Gen 0Gen 1Gen 2Allocated
JsonSerialization10,816.7 us34.447 us32.221 us6000012625.0000640.6250562.50008890.69 KB
JsonDeserialization9,769.5 us68.712 us64.273 us1109.3750312.5000312.50003760.65 KB
KrakenSerialization2,134.7 us8.597 us7.621 us300033363.2813320.3125320.31251323.63 KB
KrakenDeserialization1,061.7 us5.576 us5.216 us105.4688105.4688105.4688392.13 KB
KrakenSerializationWithOmittedRootType2,153.8 us14.298 us13.374 us300014363.2813320.3125320.31251323.68 KB
KrakenDeserializationWithOmittedRootType1,061.1 us3.685 us3.447 us103.5156103.5156103.5156392.4 KB
BinaryFormatterSerialization617.7 us10.173 us9.516 us400028255.8594213.8672212.89061427.62 KB
BinaryFormatterDeserialization188.8 us3.227 us3.018 us73.730571.777371.5332397.46 KB
NetSerializerSerialization2,058.6 us17.642 us16.502 us300004363.2813324.2188324.21881317.26 KB
NetSerializerDeserialization1,192.7 us5.112 us4.781 us103.5156103.5156103.5156391.35 KB

This benchmark serialize and deserialize an array of 100000 doubles.


BenchmarkDotNet=v0.10.14, OS=Windows 8.1 (6.3.9600.0)
Intel Core i5-4690 CPU 3.50GHz (Haswell), 1 CPU, 4 logical and 4 physical cores
Frequency=14318180 Hz, Resolution=69.8413 ns, Timer=HPET
  [Host]     : .NET Framework 4.7 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.2563.0
  DefaultJob : .NET Framework 4.7 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.2563.0
MethodMeanErrorStdDevSize (bytes)Gen 0Gen 1Gen 2Allocated
JsonSerialization57,815.4 us200.050 us177.340 us12000015875.0000625.0000500.000021681.33 KB
JsonDeserialization25,745.7 us96.432 us90.203 us2593.7500281.2500281.25009862.13 KB
KrakenSerialization6,315.2 us27.008 us25.263 us900034343.7500304.6875304.68752931.7 KB
KrakenDeserialization3,456.8 us17.746 us16.599 us78.125078.125078.1250782.28 KB
KrakenSerializationWithOmittedRootType6,390.1 us55.197 us51.631 us900014398.4375367.1875359.37502931.92 KB
KrakenDeserializationWithOmittedRootType3,441.7 us18.754 us17.542 us70.312570.312570.3125782.52 KB
BinaryFormatterSerialization1,201.8 us22.764 us23.377 us800028289.0625248.0469246.09382849.01 KB
BinaryFormatterDeserialization341.5 us6.565 us7.297 us85.449283.496183.4961788.1 KB
NetSerializerSerialization5,513.0 us32.687 us28.976 us900004390.6250351.5625351.56252927.22 KB
NetSerializerDeserialization3,170.1 us11.463 us10.722 us82.031382.031382.0313781.86 KB

This benchmark serialize and deserialize an array of 100000 decimals.


BenchmarkDotNet=v0.10.14, OS=Windows 8.1 (6.3.9600.0)
Intel Core i5-4690 CPU 3.50GHz (Haswell), 1 CPU, 4 logical and 4 physical cores
Frequency=14318180 Hz, Resolution=69.8413 ns, Timer=HPET
  [Host]     : .NET Framework 4.7 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.2563.0
  DefaultJob : .NET Framework 4.7 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.2563.0
MethodMeanErrorStdDevSize (bytes)Gen 0Gen 1Gen 2Allocated
JsonSerialization26.616 ms0.0935 ms0.0874 ms12000012812.5000312.5000250.000012.78 MB
JsonDeserialization18.051 ms0.0895 ms0.0793 ms1468.7500437.5000437.50008.58 MB
KrakenSerialization5.550 ms0.0499 ms0.0467 ms7000351609.3750359.3750296.87506.48 MB
KrakenDeserialization6.457 ms0.0421 ms0.0394 ms117.1875117.1875117.18751.53 MB
KrakenSerializationWithOmittedRootType5.602 ms0.0523 ms0.0489 ms7000141578.1250289.0625265.62506.48 MB
KrakenDeserializationWithOmittedRootType6.444 ms0.0455 ms0.0426 ms132.8125132.8125132.81251.53 MB
BinaryFormatterSerialization32.735 ms0.3129 ms0.2927 ms12000282812.5000312.5000250.000012.78 MB
BinaryFormatterDeserialization44.297 ms0.3551 ms0.3322 ms2500.0000--9.16 MB
NetSerializerSerialization5.725 ms0.0642 ms0.0600 ms7000041609.3750359.3750296.87506.48 MB
NetSerializerDeserialization4.250 ms0.0284 ms0.0252 ms93.750093.750093.75001.53 MB

This benchmark serialize and deserialize an Dictionary of int,int with 100000 items.


BenchmarkDotNet=v0.10.14, OS=Windows 8.1 (6.3.9600.0)
Intel Core i5-4690 CPU 3.50GHz (Haswell), 1 CPU, 4 logical and 4 physical cores
Frequency=14318180 Hz, Resolution=69.8413 ns, Timer=HPET
  [Host]     : .NET Framework 4.7 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.2563.0
  DefaultJob : .NET Framework 4.7 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.2563.0
MethodMeanErrorStdDevSize (bytes)Gen 0Gen 1Gen 2Allocated
JsonSerialization33.539 ms0.3623 ms0.3212 ms24000014875.0000437.5000312.500024.03 MB
JsonDeserialization27.163 ms0.2037 ms0.1806 ms1843.7500375.0000250.000010.34 MB
KrakenSerialization7.460 ms0.0704 ms0.0658 ms1000073210.9375171.8750171.87502.96 MB
KrakenDeserialization7.624 ms0.0823 ms0.0730 ms226.5625164.0625148.43755.76 MB
KrakenSerializationWithOmittedRootType7.425 ms0.0422 ms0.0353 ms1000014195.3125156.2500156.25002.96 MB
KrakenDeserializationWithOmittedRootType7.805 ms0.1120 ms0.1047 ms234.3750171.8750156.25005.76 MB
BinaryFormatterSerialization116.046 ms0.4676 ms0.4374 ms17013357000.0000187.5000125.000026.99 MB
BinaryFormatterDeserialization278.718 ms2.3552 ms2.0878 ms6687.50002875.0000937.500038.15 MB
NetSerializerSerialization7.345 ms0.0538 ms0.0477 ms1000005250.0000210.9375210.93753.72 MB
NetSerializerDeserialization6.034 ms0.0173 ms0.0162 ms164.0625164.0625164.06252.84 MB

This benchmark serialize and deserialize an Dictionary of string,int with 100000 items.


BenchmarkDotNet=v0.10.14, OS=Windows 8.1 (6.3.9600.0)
Intel Core i5-4690 CPU 3.50GHz (Haswell), 1 CPU, 4 logical and 4 physical cores
Frequency=14318180 Hz, Resolution=69.8413 ns, Timer=HPET
  [Host]     : .NET Framework 4.7 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.2563.0
  DefaultJob : .NET Framework 4.7 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.2563.0
MethodMeanErrorStdDevSize (bytes)Gen 0Gen 1Gen 2Allocated
JsonSerialization22.03 ms0.1132 ms0.1059 ms13777812250.0000250.0000187.500011.42 MB
JsonDeserialization45.00 ms0.5256 ms0.4659 ms1250.0000562.5000250.000014.16 MB
KrakenSerialization42.46 ms0.3940 ms0.3685 ms1180708437.5000375.0000375.000013.19 MB
KrakenDeserialization63.46 ms0.5791 ms0.5417 ms3250.00001250.0000500.000024.68 MB
KrakenSerializationWithOmittedRootType42.26 ms0.4422 ms0.4136 ms1180648562.5000500.0000500.000013.19 MB
KrakenDeserializationWithOmittedRootType63.65 ms0.8864 ms0.7857 ms3375.00001125.0000500.000024.66 MB
BinaryFormatterSerialization151.19 ms0.5978 ms0.5592 ms23902306937.5000687.5000500.000043.44 MB
BinaryFormatterDeserialization456.80 ms7.8719 ms7.3634 ms8375.00003250.00001000.000052.31 MB
NetSerializerSerialization11.92 ms0.0705 ms0.0625 ms980639265.6250234.3750234.37504.46 MB
NetSerializerDeserialization22.55 ms0.2004 ms0.1776 ms812.5000468.7500218.75008.24 MB

This benchmark serialize and deserialize a string of 1000 characters.


BenchmarkDotNet=v0.10.14, OS=Windows 8.1 (6.3.9600.0)
Intel Core i5-4690 CPU 3.50GHz (Haswell), 1 CPU, 4 logical and 4 physical cores
Frequency=14318180 Hz, Resolution=69.8413 ns, Timer=HPET
  [Host]     : .NET Framework 4.7 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.2563.0
  DefaultJob : .NET Framework 4.7 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.2563.0
MethodMeanErrorStdDevSize (bytes)Gen 0Allocated
JsonSerialization2.762 us0.0302 us0.0282 us10122.39947.38 KB
JsonDeserialization3.177 us0.0183 us0.0162 us0.66382.05 KB
KrakenSerialization5.087 us0.0322 us0.0301 us11261.43434.43 KB
KrakenDeserialization1.869 us0.0148 us0.0138 us1.31804.06 KB
KrakenSerializationWithOmittedRootType4.314 us0.0196 us0.0184 us10151.29703.99 KB
KrakenDeserializationWithOmittedRootType1.647 us0.0168 us0.0149 us1.17113.6 KB
BinaryFormatterSerialization5.241 us0.0478 us0.0424 us12813.01369.27 KB
BinaryFormatterDeserialization4.277 us0.0822 us0.0914 us2.80768.63 KB
NetSerializerSerialization1.846 us0.0361 us0.0443 us10050.95182.93 KB
NetSerializerDeserialization1.328 us0.0169 us0.0158 us1.32184.06 KB

This benchmark serialize and deserialize a large struct.


BenchmarkDotNet=v0.10.14, OS=Windows 8.1 (6.3.9600.0)
Intel Core i5-4690 CPU 3.50GHz (Haswell), 1 CPU, 4 logical and 4 physical cores
Frequency=14318180 Hz, Resolution=69.8413 ns, Timer=HPET
  [Host]     : .NET Framework 4.7 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.2563.0
  DefaultJob : .NET Framework 4.7 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.2563.0
MethodMeanErrorStdDevSize (bytes)Gen 0Allocated
JsonSerialization985.8 ns10.473 ns9.796 ns471.86735880 B
JsonDeserialization50,096.5 ns505.951 ns473.267 ns0.30521112 B
KrakenSerialization2,056.0 ns11.411 ns10.116 ns2330.48451536 B
KrakenDeserialization1,207.7 ns6.076 ns5.387 ns0.40051264 B
KrakenSerializationWithOmittedRootType1,150.9 ns6.535 ns5.793 ns520.33761064 B
KrakenDeserializationWithOmittedRootType855.5 ns4.080 ns3.817 ns0.1879592 B
BinaryFormatterSerialization5,317.2 ns46.628 ns41.334 ns4562.18206864 B
BinaryFormatterDeserialization6,153.7 ns41.805 ns39.104 ns2.25077104 B
NetSerializerSerialization333.0 ns1.280 ns1.069 ns410.1445456 B
NetSerializerDeserialization299.1 ns3.302 ns3.088 ns0.0405128 B

This benchmark serialize and deserialize a small class used by the Wire project.


BenchmarkDotNet=v0.10.14, OS=Windows 8.1 (6.3.9600.0)
Intel Core i5-4690 CPU 3.50GHz (Haswell), 1 CPU, 4 logical and 4 physical cores
Frequency=14318180 Hz, Resolution=69.8413 ns, Timer=HPET
  [Host]     : .NET Framework 4.7 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.2563.0
  DefaultJob : .NET Framework 4.7 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.2563.0
MethodMeanErrorStdDevSize (bytes)Gen 0Allocated
JsonSerialization2,797.8 ns24.063 ns20.094 ns1242.03706426 B
JsonDeserialization2,282.5 ns26.583 ns24.866 ns0.32041016 B
KrakenSerialization2,306.4 ns16.392 ns15.333 ns1430.43871392 B
KrakenDeserialization1,375.9 ns6.833 ns5.706 ns0.37571184 B
KrakenSerializationWithOmittedRootType1,612.6 ns9.370 ns8.764 ns500.36051136 B
KrakenDeserializationWithOmittedRootType1,130.8 ns10.235 ns9.073 ns0.2556808 B
BinaryFormatterSerialization8,616.4 ns46.589 ns41.300 ns3932.45677736 B
BinaryFormatterDeserialization9,820.7 ns111.603 ns104.394 ns2.71618568 B
NetSerializerSerialization639.5 ns3.292 ns2.918 ns390.1545488 B
NetSerializerDeserialization606.4 ns6.021 ns5.633 ns0.0677216 B