Skip to content
ixfx

Data processing

ixfx has three idiosyncratic patterns for processing data:

When to use which?

Data.Process

Data.Process is the simplest approach. Operators take a single input value and output a single value.

This is best when your processing is:

  • localised: you can and want to keep the processing operations together in the same code location, such as a function
  • single value: you’re doing a series of transformations to a single value
  • synchronous: each operation completes synchronously and the operations happen in serial
import { Process } from 'https://unpkg.com/ixfx/dist/data.js';


// Create a flow that first converts to upper case, and then
// returns true/false if first letter is a capital A
const p = Process.flow(
 value => value.toUpperCase(),
 value => value.at(0) === 'A')
);
p('apple'); // _true_

Data.Process includes some basic operators (such as average, max/min, tally, sum, rank). Since operators are just simple functions, it’s very easy to use your own, as shown in the example above, where use two custom operators.

We can set up the flow once, and then use the flow when we have data, for example in an event handler or when iterating using for of:

const coordFlow = Process.flow(
  value => ({ x: value.x, y: value.y }), // Pluck just x,y
  value => Points.divide(value, window.innerWidth, window.innerHeight) // Make relative to viewport
)
document.addEventListener(`pointermove`, event => {
  const relativePoint = coordFlow(event);
  // Gets a relative { x, y }
})

Chains

Chains are similar to Data.Process but has generators as the basis for processing. Each link of the chain takes in a generator and returns a generator. This means it can work with sources that are potentially infinite - for example a generator that returns a random number each time it is used.

Chains are also lazy in that they don’t request or process values until the chain is actually consumed (eg. with a for of loop).

Chains make most sense when you have a flow of data processing which is:

  • localised: you can and want to keep the processing operations together in the same code location, such as a function
  • based on an iterable or generator: this could be a collection such as an array, but also potentially infinite generators which produce values from user input.

In this example, we create a generator that yields a relative pointer coordinate:

const relativePoints = Chains.run(
  Chains.From.event(document, `pointermove`),
  Chains.Links.transform(v => ({ x: v.x, y:v.y }),
  Chains.Links.transform(v => Points.divide(v, window.innerWidth, window.innerHeight))
);

To use the generator, we have all the usual techniques.

Like Data.Process, the generator-based pattern of Chains seems most ergonomic when processing data in a localised kind of way. Elegant if you want to consume the output of a fixed set of data:

const someChain = Chains.run(/** omitted */);
for await (const v of someChain) {
  // Do so something with each value...
}

…but seems less sensible for the pointermove example above.

Rx

Rx is short for ‘reactive’, ixfx’s signal-type pattern. It mostly resembles events, in how a source can emit values when it likes, and one or more receives subscribe to its data.

Like events, reactives typically don’t do any work unless they have a subscriber.

Reactives are best used when:

  • Data processing is spread out, and you want to have decoupled producers/consumers of data
  • A stream, or event-oriented model is preferable over iteration or a single value
  • Want more possibility to split up, merge and direct streams of data.
// First set it up
const relativePoints = Rx.run(
  Rx.From.event(document, `pointermove`, {x:0,y:0}),
  Rx.Ops.transform(v => ({ x:v.x, y:v.y })),
  Rx.Ops.transform(v => Points.divide(v, window.innerWidth, window.innerHeight))
);


// Subscribe to values - triggered when a move happens
relativePoints.onValue(v => {
  // { x:number, y:number }
})

As you can see, ixfx tries to keep the syntax very similar between Rx and Chains.

An advantage of the reactive pattern is that you can un/subscribe whenever and where ever in your code you like. The data source and subsequent processing becomes a unit, and the rest of your code doesn’t need to know its details.

Reactives offer more possibility to control their flow at a higher-level, merging them, splitting them etc.

Some reactives also offer the possibility to pull a value from them, these are called ‘pingable’ reactives.