Extremely Fast

December 24, 2016 ยท View on GitHub

csvtojson takes care of performance and optimise for Node.js apps.

This project shows csvtojson is about 4 - 6 times faster than other popular csv parsing libraries.

Performance Optimisation

Version 1.1.0 is much faster than versions before.

Below test is parsing a 300k records csv on a 4 core machine

TimeworkerNum = 1workerNum = 2 (fork)*workerNum = 3workerNum = 4
1.0.311.806s15.945s8.611s8.314s
1.1.09.707s10.065s5.955s4.563s

*when workerNum=2, it only creates 1 extra worker to unblock main process. It is reasonalbe it is slightly slower than workerNum=1 (just use main process).

The result shows V1.1.0 has about 30% - 50% performance boost.

CPU usage leverage

Below is CPU core usages for v1.1.0 when running the test:

CoreworkerNum = 1workerNum = 2workerNum = 3workerNum = 4
Core 1 (Main)100%25%60%90%
Core 2N/A80%70%70%
Core 3N/AN/A70%70%
Core 4N/AN/AN/A70%