Transforms

January 26, 2025 · View on GitHub

A transform is a function to preprocess data before it is converted into CSV. Each transform receives each data record, performs some processing and returns a transformed record.

Built-in transforms

There is a number of built-in transform provided by this package.

Installation

NPM

You can install json2csv transforms as a dependency using NPM.

$ npm install --save @json2csv/transforms

import { unwind, flatten } from '@json2csv/transforms';

Yarn

You can install json2csv transforms as a dependency using Yarn.

$ yarn add --save @json2csv/transforms

import { unwind, flatten } from '@json2csv/transforms';

CDN

json2csv transforms are packaged as an ES6 modules. If your browser supports modules, you can load json2csv transforms directly on the browser from the CDN.

You can import the latest version:

<script type="module">
  import unwind from 'https://cdn.jsdelivr.net/npm/@json2csv/transforms';
</script>

You can also select a specific version:

<script type="module">
  import unwind from 'https://cdn.jsdelivr.net/npm/@json2csv/transforms@7.0.7';
</script>

Unwind

The unwind transform deconstructs an array field from the input item to output a row for each element. It's similar to MongoDB's $unwind aggregation.

The transform needs to be instantiated and takes an options object as arguments containing:

paths <String[]> List of the paths to the fields to be unwound. It's mandatory and should not be empty.
blankOut <Boolean> Flag indicating whether to unwind using blank values instead of repeating data or not. Defaults to false.

Examples

Simple unwind

Parser

import { Parser } from '@json2csv/plainjs';
import { unwind } from '@json2csv/transforms';

const data = [
  { "carModel": "Audi", "price": 0, "colors": ["blue","green","yellow"] },
  { "carModel": "BMW", "price": 15000, "colors": ["red","blue"] },
  { "carModel": "Mercedes", "price": 20000, "colors": "yellow" },
  { "carModel": "Porsche", "price": 30000, "colors": ["green","teal","aqua"] },
  { "carModel": "Tesla", "price": 50000, "colors": []}
];

try {
  const opts = {
    transforms: [
      unwind({ paths: ['colors'] })
    ]
  };
  const parser = new Parser(opts);
  const csv = parser.parse(data);
  console.log(csv);
} catch (err) {
  console.error(err);
}

Stream Parser

import { StreamParser } from '@json2csv/plainjs';
import { unwind } from '@json2csv/transforms';

const data = [
  { "carModel": "Audi", "price": 0, "colors": ["blue","green","yellow"] },
  { "carModel": "BMW", "price": 15000, "colors": ["red","blue"] },
  { "carModel": "Mercedes", "price": 20000, "colors": "yellow" },
  { "carModel": "Porsche", "price": 30000, "colors": ["green","teal","aqua"] },
  { "carModel": "Tesla", "price": 50000, "colors": []}
];

const opts = {
  transforms: [
    unwind({ paths: ['colors'] })
  ]
};
const parser = new StreamParser(opts, { objectMode: true });

let csv = '';
parser.onData = (chunk) => (csv += chunk.toString()));
parser.onEnd = () => console.log(csv));
parser.onError = (err) => console.error(err));

data.forEach(record => parser.write(record));

Node Transform

import { createReadStream, createWriteStream } from 'fs';
import { Transform } from '@json2csv/node';
import { unwind } from '@json2csv/transforms';

const data = [
  { "carModel": "Audi", "price": 0, "colors": ["blue","green","yellow"] },
  { "carModel": "BMW", "price": 15000, "colors": ["red","blue"] },
  { "carModel": "Mercedes", "price": 20000, "colors": "yellow" },
  { "carModel": "Porsche", "price": 30000, "colors": ["green","teal","aqua"] },
  { "carModel": "Tesla", "price": 50000, "colors": []}
];

const input = createReadStream(inputPath, { encoding: 'utf8' });
const output = createWriteStream(outputPath, { encoding: 'utf8' });
const opts = {
  transforms: [
    unwind({ paths: ['colors'] })
  ]
};
const parser = new Transform(ops);

const processor = input.pipe(parser).pipe(output);

Node Async Parser

import { AsyncParser } from '@json2csv/node';
import { unwind } from '@json2csv/transforms';
import { addCounter } from './custom-transforms';

const opts = {
  transforms: [
    unwind({ paths: ['colors'] })
  ]
};
const parser = new AsyncParser(opts);

let csv = await parser.parse(data).promise();

WHATWG Transform Stream

import { TransformStream } from '@json2csv/whatwg';
import { unwind } from '@json2csv/transforms';

const opts = {
  transforms: [
    unwind({ paths: ['colors'] })
  ]
};
const parser = new TransformStream(opts);

await sourceStream.pipeThrough(parser).pipeTo(writableStream);

CLI

At the moment, only built-in transforms are supported by the CLI interface.

$ json2csv -i data.json --unwind "color"

// Unwind a single field
unwind({ paths: ['fieldToUnwind'] });

// Unwind a single field and blank out repeated data
unwind({ paths: ['fieldToUnwind'], blankOut: true });

Flatten

Flatten nested JavaScript objects into a single level object.

The transform needs to be instantiated and takes an options object as arguments containing:

objects <Boolean> Flag indicating whether to flatten JSON objects or not. Defaults to true.
arrays<Boolean> Flag indicating whether to flatten Arrays or not. Defaults to false.
separator <String> Separator to use between the keys of the nested JSON properties being flattened. Defaults to ..

// Default
flatten();

// Custom separator '__'
flatten({ separator: '_' });

// Flatten only arrays
flatten({ objects: false, arrays: true });

Custom transforms

Users can create their own transforms as simple functions.

function doNothing(item) {
  // apply tranformations or create new object
  return transformedItem;
}

or using ES6

const doNothing = (item) => {
  // apply tranformations or create new object
  return transformedItem;
};

For example, let's add a line counter to our CSV, capitalize the car field and change the price to be in Ks (1000s).

function addCounter() {
  let counter = 1;
  return (item) => ({
    counter: counter++,
    ...item,
    car: item.car.toUpperCase(),
    price: item.price / 1000,
  });
}

The reason to wrap the actual transform in a factory function is so the counter always starts from one and you can reuse it. But it's not strictly necessary.

How to use transforms

Transforms are added to the transforms option when creating a parser. They are applied in the order in which they are declared.