What is Perspective?
Perspective is an open source streaming analytics engine from JP Morgan. It uses WebAssembly to quickly analyse a dataset and create views for displaying in a chart or grid. Perspective also has a UI which you can use to pivot and filter the data, and uses a plugin framework to view the results in grid or chart form. This is where d3fc comes in.
Perspective already has a Highcharts plugin for the chart views. It looks pretty good, but Highcharts requires a paid license, so isn’t suitable for everyone. The d3fc library seemed like a perfect fit as an open source replacement for Highcharts.
Here’s an example of the d3fc plugin in action:
Or click here for a live example you can interact with: d3fc Example
d3fc as a generic charting view
If you have a look at the d3fc code examples, you’ll find that each one is designed for a specific chart view. They have data that is structured for the intended view, along with axes, series and other features that are customised for that view. None of that really lends itself to a generic chart component that can display a wide variety of charts.
We want to follow the style of D3 and d3fc components, and build up a set of tools that can be reused across the various chart types.
The entry point for rendering a chart will just be a simple function, where
settings contains the data, and also all the configuration information (what the row and column splits are, and field types etc).
Data and Series
Each chart type will need its own representation of the data to match the type of series we want to use. For the column chart in the image above, we need multiple
seriesSvgBar (one for “Sales” and one for “Profit”), combined with
seriesSvgMulti. If you pick something in the “Split By” option, then we also want to stack bars on top of each other, so to do that we need
Other chart types will need to re-use a lot of this code. The horizontal bar chart is identical to the column chart, whereas the line and area charts are very similar, but without the stacking.
I’ll leave the details of the data function to your imagination (or else you could go and look at the source). Essentially it’s taking a simple table of data and re-writing it into an array for each series.
In the above example, we have an ordinal horizontal axis (categories) and a linear vertical axis (numeric). However, if instead of picking “abc” (string) values for the “Group By” setting, you choose a date value, the chart will display a linear horizontal axis (datetime).
A lot of other chart types will have similar requirements for axis types, so we want to move this logic into a reusable component. Some of those charts can also support displaying an ordinal Y axis, but for the column chart it doesn’t make any sense to draw a column between two ordinal values.
Our X/Y Scatter chart actually prefers linear axes for both X and Y (though does actually support an ordinal axis in both directions too). The Heatmap chart is the opposite as it’s designed primarily for ordinal axes, but can also display linear.
In the abbreviated example above, “crossValues” and “mainValues” identify the values in the source data that the axis will be based on, so the
axisFactory will check what type those values are (ordinal / datetime / linear). Internally it composes factories for each of those three types that can deliver the correct scale, domain and other settings.
The value returned from
axisFactory is actually an object that contains the scale and domain, along with a label function, sizing information, and sometimes a custom component to change how the axis is rendered (more on that later). Putting together the two sets of axis objects into a chart is going to be similar across all chart types, so we move that into a factory component too.
As part of this project, we’ve made a few contributions back to d3fc. I hope to have a follow-up post soon that will go into those enhancements in a bit more detail. One of them, which can be seen in the above example, allows us to provide the cartesian chart with a custom axis component.
We’ve implemented a custom axis component that can split the axis labels into groups, and automatically rotate and hide labels when there is not enough room to display them all horizontally. I hope to write a follow-up on this as well, with a working example of a custom axis that can easily be reused on other projects. Here’s another example:
Our initial objectives with this project were simply to offer like-for-like replacements for the Highcharts views that d3fc supports. Later, that expanded to include other non-cartesian chart types that required a bit more D3 magic (“sunburst” and “treemap”). We’re now getting close to a complete, open source replacement for the Highcharts plugin.
For bonus points, we’ve even added an extra chart type especially for financial data, showing an OHLC/Candlestick view with bollinger bands and moving average.
The Perspective d3fc plugin is now a core plugin in Perspective. You can see the package here: d3fc Plugin Project
Colin has previously explored building custom d3fc visualisations on realtime data with Perspective. You can read that here: Realtime crypto charting with JPM Perspective and d3fc.