Working With D3.js

21 Nov 2014 Working With D3.js

D3.js is a JavaScript library that is used to generate visualisations, usually based on sets of data. I have been using it on a dashboard project for the past month, and wanted to share some of my thoughts and impressions as a new user.

First Impressions

The first thing that appealed to me about using D3 is that it utilises the available components of modern browsers (HTML5, Javascript, CSS, SVG) without requiring the installation of a plug-in to make it work.

As you are developing these visualisations, you can see all the components you are creating and manipulating right there in the DOM. Using Chrome dev tools to inspect the DOM as you work means it is quite easy to see what is going on as you learn to use D3, and see where you are going wrong.

Since you are dealing with DOM elements, you can take full advantage of CSS, keeping your javascript code relatively clean.

Another big plus for me was the similarity to jQuery with its selectors and operators. Having used jQuery for some time (and finding that a revelation in itself when I discovered it) the familiarity definitely helped with the learning process.

So, technically, D3.js provides you with a great tool for producing brilliantly beautiful visualisations that will greatly impress. However, there are a few things to consider before diving head-first into this stunning world.

Specifically, there are a some concepts and tricky parts that seem strange at first and require a bit of a leap-of-faith before everything falls into place. Although the methods can be utilised individually, the results are not really visible until many are chained together.

Getting Technical

I won’t go into a full example, but will attempt to give you a feel for what to expect.

Say you want to produce a bar chart to represent a dataset.
First you bind the data to a DOM selector, then chain the enter() command which iterates over the data, and then chain the append() command to create the elements that match the original selector. The code may look something like this:

var svg = d3.select("svg");

var bars = svg.selectAll("rect")
    .data(dataset)
    .enter()
    .append("rect")
    .attr({ x: …,
            y: …,
            height: …, // height based on dataset values.
            width: … });

Then, be amazed at what happens when updating the values in the dataset:

bars.data(newDataset)
.transition()
.duration(2000)
.attr({ height: … }); // new height based on newDataset values.

Like magic, the chart will change before your eyes. It really was a “wow” moment for me.

That said, I would recommend using known test data when building up your initial charts, in order to make it more obvious when the result is right or wrong.

Another aspect which may be confusing is the generation of lines and paths. This is mainly because the HTML output data for paths is in a format that is not easy to read. For example, a relatively simple line chart might contain a path that looks like this:

<path d="M75,82.40274599542334Q129.4,79.8350114416476,143,79.25858123569793C163.4,78.39393592677345,190.6,75.30217391304348,211,76.63844393592677S258.6,87.45961098398168,279,88.16704805491989S326.6,84.65606407322653,347,81.35469107551486S394.59999999999997,68.04439359267735,415,66.15789473684211S462.6000000000001,66.97013729977117,483.00000000000006,68.77803203661327S530.6,76.16681922196797,551,78.21052631578948S598.6,82.79576659038902,619,82.40274599542334S666.6,77.00526315789476,687,75.59038901601832Q700.6,74.6471395881007,755,72.97025171624713" stroke="#CCCCCC" stroke-width="2"></path>

Help Me!

D3 provides a number of helpers to assist with getting positioning and sizing right with your creations, however, a decent grasp of geometry was certainly of benefit to me during development.

The X,Y coordinate system in D3 starts at (0,0) in the top left corner of your page, with X increasing to the right, and Y increasing down the page. Compare this with your standard bar-chart which usually has (0,0) at the bottom left – so essentially the y-axis is turned upside-down.

Thankfully D3 comes to the rescue with the handy scale() function. This helps translate your data points to pixel points:

var myXScale = d3.scale.linear()
    .domain([0, 100])
    .range([20, 400]);

where the domain is the minimum and maximum input from your dataset, and range is the minimum and maximum output point to draw on the page. So passing in the middle data point would give you the middle pixel point. eg. myXScale(50) returns 210.

The clever part is that in order to map your data to the “upside-down” Y-axis, all you need to do is reverse the range values. It could look something like this:

var myYScale = d3.scale.linear()
    .domain([0, 100])
    .range([400, 20]);

You can see how the smaller data values will map to lower points on the page, and larger values to higher points on the page – just like you would expect on most charts.

Even though these helpers are provided, having lots of number ranges and associated (x,y) (or (x1,y1), (x2,y2)) attributes to deal with can stretch your capacity for mental arithmetic… or you can always resort to trial-and-error. (After all, a simple page refresh will immediately show you the result of your changes.)

I have found the documentation of D3 to be a little unfriendly as well. However, the vast pool of examples and tutorials available on the web more that makes up for this.

Another thing I had some trouble with presenting a map of Australia with highlighted points based on latitude/longitude coordinates. Using GeoJSON data seemed like the way to go, and after fumbling my way to getting the map looking right with a suitable projection (refer to d3.geo), my function for translating lat-long to pixel coordinates resulted in some points landing in the ocean. I eventually discovered that the projection used to plot the map on the page is actually a translation of lat-long to pixels coordinates itself – and I could re-use the projection for my coordinate mapping. The result was pixel-perfect of course!

Iteration, Iteration, Iteration

Although D3 is a great tool, it does not guarantee the result will look fantastic straight away. A good design is still required and, because it works at such a low level, it may take several iterations to get it looking right. Consider the following iterations of a line chart:

After creating the basic line plot above, the second iteration has added circles at the data points, and the third has the data values at each point. The code for each addition feature (e.g. a circle) is written once (as a function of the data values), but applied to all data points within the dataset. Since the feature is a function of the data, changes to the dataset automatically applies the change to the feature on each point.

I was initially building the visualisations for a large, high-res screen. However it was found that being able to present on smaller tablet screens would also be useful. The resolution change did not play well with the design at first, but refining the design and incorporating the scaling functions across all components of the charts resulted in a more flexible model.

Conclusion

It you want good results without the design effort, you may want to look at alternatives like Chart.js. Considering we initially started with Chart.js and produced some reasonably attractive charts, the decision to use D3 was driven by the glorious productions of our graphic designer.

In short, the need for low-level detail pushed us in the direction of D3, but the extra development effort required is rewarded by almost infinite flexibility and control.