JP Morgan’s Perspective is an open source streaming analytics engine that uses WebAssembly to quickly analyse a dataset and display views via a plugin framework.
In Building a Perspective plugin with d3fc, I looked at how we used d3fc to render a generic graph view. In this post, I want to take a closer look at the Plugin API.
For this example I wanted something a bit different than the usual charts and grids, but with some visual appeal, so I have gone for a mapping plugin based on OpenLayers and OpenStreetMap.
Here is a working example that you can interact with: Perspective Maps Example. Try dragging the
region variable into “Split By” or “Filter” to see how you can manipulate the view.
The completed project is on GitHub: perspective-viewer-maps.
NYC Citibike data based on Andrew Stein’s NYC Citibike Analytics blog: Perspective Maps with Citibike.
Map of airports on OpenFlights: Perspective map of world airports.
Creating a new plugin project is really easy. I have created a starter template for this purpose: perspective-viewer-template. I should note that this is my own template, not an official Perspective one, though hopefully sometime soon there will be an official option to get started with.
Copy it into a new project
Simply clone or copy the template into a new project (with a different name), and update the name in
package.json. I called my example “perspective-viewer-maps”.
Install the dependencies and run it:
That’s it! The template comes with a default view that renders to a table:
About the template
The default view is in
/src/js/views/view-1.js. New views should be added to the list in
A view is a function that takes
Next, I added a dataset to the included
examples folder (Met Office sites with latitude/longitude), and updated
index.html to show the new data.
longitude aggregates use
avg rather than
sum, since adding them together makes no sense. An average will give us the average position when grouping.
This view uses the included X/Y Scatter chart to show that we’ve loaded the data correctly.
So let’s show an actual map.
Configure the plugin’s view
Rename the template’s default view (“view-1”) to “map-view”, and give it some default parameters so that the user is prompted for “Longitude” and “Latitude”:
Add OpenLayers with OpenStreetMap
Add OpenLayers to the project (
npm install ol), and update the view function to render a simple map into the container:
Bundle the CSS for OpenLayers
Perspective uses custom elements with shadow-DOM, which means that any CSS loaded at the document level will not apply to the plugin. However, we can bundle OpenLayers’ css with the plugin’s css by adding this line to the less file (
Plot the data
We need to turn the data from Perspective into a list of points. I’ve omitted the implementation of
getMapData() from this example for brevity, and also because it’s going to change quite a lot later (when we start dealing with group-by and split-by options). For now it’ll be enough to take each row of the data, get the
latitude values from it, and turn it into a “feature”.
At this point we need to take a step back and sort out a few things I’ve skipped over. I’ll try to be brief as the full implementation is available in the
Deal with grouped and split data
When the user drags a variable into the “Group By” box, Perspective will give us rows with a
__ROW_PATH__ array (with a value for each group variable), and additional rows for the totals. For this plugin we’re not using the totals, so we filter out any row where
__ROW_PATH__.length is less than the number of group variables (
When the user drags a variable into the “Split By” box, Perspective gives us rows where the values use a combined key of the split-by variables and the aggregate. e.g. if I use “region”, I’d see values like “sw|latitude” for the latitude value of the “sw” split, and “sw|longitude” etc… We’ll process these values and give each point a “category” label that we’ll use later to colour the points.
See the completed
getMapData() function for full details.
Re-use the map object
At the moment it’s clearing the display and rendering a new map each time, so we need to re-arrange the code a little so that we can store the
VectorLayer as private data associated with the DOM node. We can then re-use them when the view is updated, and re-populate the
VectorLayer with the new points.
Work out the initial view extents
My first example used a hard-coded location to show the whole of the UK. That works for my sample data, but wouldn’t be much use for anywhere else. We need a function that enumerates through the data to get minimum and maximum longitude and latitude. When we initialise the map view the first time, we can set the
resolution to give us a suitable initial viewing area.
Here are a few more features that I have added, and some things that I intend to continue to work on.
Colour based on “Split By” categories
When the user has picked a “Split By” value, we want to colour the points on the map based on which split / category they fall into.
I added an array of category colour values to the code. When there is a category available, we associate each category with one of the colours, and use that colour to style the circle on the map.
Size based on a third aggregate value
I’ve added a third (optional) variable to the plugin’s configuration to specify size:
When creating features for the map, we can use the “Size” value to set the feature’s size. We’ll need to enumerate the data to work out the minimum and maximum sizes, and also use a suitable default size as a fallback.
Tooltips and Click events
mouseover event, we just need to find the closest point and show a tooltip div with the associated data.
A click event is similar, but we need to send the associated data up to the hosting application to let it know what was clicked. We can do this using a custom event:
In the event details:
column_namesrefers to the aggregate columns selected. In this case it’s all of them since you can’t click data that represents “Longitude” but not “Latitude” or “Size”.
filtersshould be the list of filters that would need to be applied to another
perspective-viewerto filter it to this data. i.e. it is the current filter combined with a filter representing the point that was clicked.
rowis the original source row given to us by
A note about bundling
Creating a new plugin for Perspective really is quick and easy. Most of my time on this project was spent trying to figure out the CORS problem with OpenLayers! The core build took very little time.
Perspective gives us a powerful platform for manipulating all sorts of data ready to display, and the plugin system means that anyone can create their own view. I’m sure there are many different ways to display data than the traditional chart and grid views.