FilterBase

All provided filters (pick, replace, ignore, filter) are built on filterBase. It is a factory that produces configurable token-stream filter functions.

filterBase operates on token streams produced by a parser or another filter. Filters go after a parser and can be chained.

API

This document describes the user-facing interface only. If you want to build your own filter, feel free to inspect the code to gain more insights.

Internally, FilterBase keeps track of objects by building a stack. Items of a stack can be:

• Number. In this case, a corresponding object is an array, and the number is the current index.
• String. In this case, a corresponding object is an object, and the string is the current property key.
• null. In this case, a corresponding object is an object, but keys are not tracked. FilterBase keeps track of keys only if a previous stream returns packed keys. In this case, a filter assumes that only object's shape will be used for filtering.

The stack is used to make filtering.

Options

All filter factories accept an options object with the following properties:

• pathSeparator is a string that separates stack values when it is converted to a string. The algorithm is straightforward: stack.join(pathSeparator). The default: '.'.

  const obj = [{a: 1}, {b: 2}];
  
  // stack when filtering 1: [0, 'a']
  // converted to a string: '0.a'
  
  // stack when filtering 2: [1, 'b']
  // converted to a string: '1.b'

• filter is a way to accept or reject a data item. The interpretation of its returned value is up to concrete filter objects. Its value can be one of the following types:

  • String. The stack is converted to a string using pathSeparator, then it should be equal to filter value, or it should be longer and the filter value should be on a boundary of the pathSeparator value.

    const obj = {a: [1, 2], ab: null};
    
    const filter = 'a';
    // it fits ['a'], ['a', 0], and ['a', 1], but not ['ab']

  • RegExp. The stack is converted to a string using pathSeparator, then the filter is applied using filter.test(path).

    const obj = {a: [1, 2], ab: null};
    
    const filter = /^a\b/;
    // it fits ['a'], ['a', 0], and ['a', 1], but not ['ab']
    
    const filter = /^a/;
    // it fits ['a'], ['a', 0], ['a', 1], and ['ab']

  • Function. The filter is applied using filter(stack, chunk), where chunk is a data item being filtered. The function is called in the context of current filter object. It should return a truthy/falsy value.

  • The default: () => true.

• once is a flag. When it is truthy, a filter object will make a selection (depending on its definition of selection) only once. Otherwise, all selections are included. The default: false.

  • It can be used as an optimization feature when we know that our stream contains exactly one object we want to do our action on.

• maxDepth is the maximum JSON nesting depth a filter evaluates. When a token is nested deeper than this, the filter throws a RangeError instead of matching its path — a guard for untrusted input with unbounded nesting. The default: 1024. Pass Infinity to remove the limit.

• replacement is what should be used instead of skipped objects. Not all filters use this option. Its value can be one of the following types:

  • Function. The filter is applied using replacement(stack, chunk), where chunk is a data item being filtered. The function is called in the context of current filter object. It should return an array of semantically valid data items.
  • Otherwise, it is assumed to be a static array of semantically valid data items.
  • The default for replace: none (the value is removed). Specify replacement explicitly to substitute with a different value.

• Streaming flags control how delayed keys are replayed. Both are for compatibility with Parser. See Parser's options.

  • streamValues is assigned first.
  • streamKeys is assigned next. When its effective value is falsy, no startKey, stringChunk, nor endKey are produced. Only keyValue is issued.
  • The default: true.

• packKeys — when true, the filter expects packed keyValue tokens from upstream. Required by replace and ignore.

Important details

Stack and path

When using a string or a regular expression as a filter function, the stack is converted to a path string before the filter can be applied. It should be noted that when a source stream does not produce keyValue data items, the stack uses null to denote an undefined property key, which is converted to a path string as an empty string:

[].join('.'); // ''
[null].join('.'); // ''
[null, null].join('.'); // '.'
[null, 1, null].join('.'); // '.1.'
[1, null, null, null, 2, null].join('.'); // '1....2.'

Be aware of this behavior when crafting filters.

Property keys can be arbitrary strings. Sometimes it can mess up paths and textual filters. In order to avoid it, you can choose a different pathSeparator. It can be any string you like, just make sure it works with your filters.

const f = filter({pathSeparator: '->'});
// it will produce paths like that:
// [1, 'a'] => '1->a'
// [1, 0, 'ab', 0] => '1->0->ab->0'

Replacement hazards

Filters do not check if an array of replacement items is valid or not. Malformed arrays will produce substreams, which can break the rest of the data pipeline. Be extra careful with the replacement option.

makeStackDiffer

Named export from stream-json/filters/filter-base.js. Returns a function that emits the structural tokens (startObject, startArray, startKey + stringChunk + endKey, endObject, endArray) required to reconstruct the surrounding JSON envelope between two stack positions. Used internally by filter and replace to bridge non-contiguous matches; exposed for consumers building custom filters on top of filterBase.

import {makeStackDiffer} from 'stream-json/filters/filter-base.js';

const differ = makeStackDiffer(/* previousStack */ []);
// differ(stack, chunk, options) → Many<Token>
// Call it on each match site; the returned tokens carry the structural
// envelope (open object/array, key tokens) needed to land `chunk` at
// the right depth in the output stream.

Arguments:

• previousStack (optional) — initial stack the differ should treat as the "already emitted" position. Defaults to [] (root). Pass the stack state from the previous emit when chaining diffs.

The returned function takes (stack, chunk, options) and returns a Many<Token> of the bridging structural tokens, where options is the filter's FilterBaseOptions (so the differ honors streamKeys, streamValues, packKeys, and pathSeparator).

This is a low-level utility — most users compose existing filters (pick, replace, ignore, filter) instead. Reach for it when writing a custom filter on top of filterBase that needs to recreate parent containers between non-adjacent matches.