Building advanced graph visualizations with D3 and KeyLines

This is the last in a series of blog posts written for D3 developers interested in network visualization.

Read part 1 and part 2

In previous posts of this series I showed how to build a basic network visualization with a force directed layout and some customized labels, arrows and glyphs.

This time, let’s continue our journey with some more advanced features!

Graph Function: Neighbors

Normally after running their layout, a user wants to make a selection and start diving into their graph. A common way to do this is by selecting and highlighting a node and its neighbors, using the mouse click event.

Let’s see how this works in D3, then KeyLines.

Select and highlight neighbors with D3

D3, unhelpfully, doesn’t have a concept of neighbors. Instead we need to build it.

The first step is to create a rectangle in the background, to capture clicks on the canvas and call the restore selection function:

var canvas = svg.append("svg:rect")
    .attr({
        "height": height,
        "width": width,
        "fill": "none"
    });
...

canvas.on("click", function() {
        deHighlight();
    });

We also bind the highlight function to the node click:

var node = svg.selectAll(".node")
    ...
    .on("click", function(d) {
        highlightNeighbors(d);
    });

Then we create a dictionary to look up neighbors of a specific node:

var neighbors = {};

force.nodes().forEach(function(node) {
    neighbors[node.index] = neighbors[node.index] || [];
})

force.links().forEach(function(link) {
    neighbors[link.source.index].push(link.target.index);
    neighbors[link.target.index].push(link.source.index);
});

Next we want to visually highlight the neighbors of the nodes we click. This again is a bit fiddly in D3. We add a new “background” class to non-neighboring item, using D3 .classed method, and control the class behavior with css: .background { opacity: 0.1; }. This will ‘ghost’ items into the background.

In the same function, we also want to display details (node and edge labels) of the highlighted items:

function highlightNeighbors(d) {

    deHighlight();

    node.classed("background", function(n) {
        return neighbors[d.index].indexOf(n.index) != -1 ? false : true;
    });

    link.classed("background", function(n) {
        return d.index == n.source.index || d.index == n.target.index ? false : true;
    });

    nodeLabels.filter(function(n) {
        return neighbors[d.index].indexOf(n.index) != -1;
    })
    // we can't use display:none with labels because we need to load them in the DOM in order to calculate the background rectangle dimensions with the getBBox function. 
    // So we used visibility:hidden instead.
    .style("visibility", "visible");

    textBackground.filter(function(n) {
        return neighbors[d.index].indexOf(n.index) != -1;
    }).style("display", "inline-block");

    linkLabels.style("display", function(n) {
        return d.index == n.source.index || d.index == n.target.index ? "inline-block" : "none";
    });

    // select self and change its properties
    d3.select(this).classed("background", false);
    d3.selectAll(this.childNodes).style({
        "display": "inline-block",
        "visibility": "visible"
    });
};

It might also be useful to add a counter glyph to the node, showing the node degree:

...
glyphLabels.text(function() {
    return neighbors[d.index].length;
});
...

Then finally, we also need a function to remove highlight settings:

function deHighlight() {
    node.classed("background", false);
    link.classed("background", false);
    linkLabels.style("display", "none");;
    nodeLabels.style("visibility", "hidden");
    nodeGlyph.style("display", "none");
    glyphLabels.style("display", "none");
    textBackground.style("display", "none");
}

After all of that coding, you might – with some luck – get a result like this:

Select and highlight neighbors with KeyLines

Happily, a comparable effect can be achieved in KeyLines using some handy functions:

chart.foreground(),
chart.selection(),
chart.getItem() and
chart.graph().neighbours()
combined in the following order:

chart.bind('click', highlightNeighbors);

var highlightNeighbors = function(id) {

    // this function looks up into the neighbors dictionary
    function areNeighboursOf(item) {
        return neighbours[item.id];
    }

    var neighbours = {};
    deHighlight();

    // select timebar lines
    tlSelect(chart.selection());

    // exclude click on canvas
    if (id != null) {

        var item = chart.getItem(id);

        // capture click on nodes
        if (item && item.type === 'node') {
            
            // get neighboring items (nodes and links)
            var result = chart.graph().neighbours(id);

            // store property changes in a variable for later use and add neighbors to the dictionary
            var changes = result.nodes.concat(result.links).map(function(id){
                neighbours[id] = true;
                var neighbor = chart.getItem(id);
                return {
                    id: neighbor.id,
                    t: neighbor.d.label
                }
            });

            // add self to the neighbors dictionary and add the glyph
            neighbours[id] = true;
            changes.push({
                id: id,
                t: item.d.label,
                g: [{
                    p: 'ne',
                    c: '#de5835',
                    w: true,
                    e: 1.3,
                    t: result.nodes.length
                }]
            });

     // chart.setProperties() allows us to change multiple properties at a time, targeting item ids (note that in deHighlight() we use it with RegEx option to target all ids!).
            chart.setProperties(changes);

            // put neighbors in the foreground
            chart.foreground(areNeighboursOf, { type: 'all' });

            // select neighboring items
            chart.selection(result.nodes.concat(item.id).concat(result.links));
        }
    }
};

var deHighlight = function() {
    chart.setProperties({ id: '.', t: null, g: null }, true);
    chart.foreground(function() { return true; }, { type: 'all' });
};

KeyLines associates a click with an item id. This is much more convenient than D3’s approach that associates it with the target visual object, as it means we can construct simple conditionals to catch canvas clicks and node clicks (or others).

The chart.graph().neighbours function returns neighbor nodes. This function accepts some very convenient, graph specific, options, e.g.:

• direction defines if neighbors should be found – inbound links, outbound links, or both
• hops defines how many levels of neighbors to find

In the code above we also used KeyLines custom data property (d), which we have previously fed with multiple fields in the parsing function:

var nodes = obj.nodes.map(function(node) {
    return {
        ...
        d: {
            label: node.Name,
            address: node.Address,
            city: node.City,
            events: [node.Event1, node.Event2]
	    ...
        },
        ...
    };
});

By contrasting the two approaches here, we can see that the KeyLines API makes some common graph visualization tasks much simpler for the developer to implement.

Advanced graph functions

In the previous section we explored the structure of our network by searching for a node’s neighbors and calculating its degree. Degree is a useful centrality measure, but can be too simplistic. Sometimes more advanced options – e.g. betweenness or PageRank, give a more useful view.

Let’s discuss how we implement these in the two toolkits.

Advanced graph functions in D3

With D3 you are left to your programming and math skills to create a graph function – and graphs tend to involve very demanding calculations. Optimization is also a big hurdle, not to mention the task of binding your custom centrality scores to a visual property or updating the graph in real-time in response to user behaviors.

Advanced graph functions in KeyLines

Happily, KeyLines has you covered.

There are plenty of native graph functions (see them on our social network analysis page) that have been designed and carefully optimized to work across virtually all use cases.

There is another benefit in using KeyLines: it has been designed only for graph visualization.

Every function has been written to work alongside the rest of the graph API, and graph theory is interwoven into the code – making it easy to layer functionality and tell a great network story.

Network Analysis is only one of the features of KeyLines. Other functionality, like automatic layouts, combos, network filtering, help users to work through dense networks and focus on the data they need to understand.

Another key task your users will probably want to do is understand temporal or geographic trends in their graphs. Let’s look at these two in more detail.

Dynamic networks

Networks are hardly ever static. They change through time, so your graph visualization needs to be able to show that. The most intuitive way for this is using a time bar component.

Dynamic networks with KeyLines

One of KeyLines’ most popular pieces of functionality is the time bar:

It is incredibly responsive, easy to use and simple to incorporate into your graph visualization app. First, we change a bit our KeyLines.create() function to create the two different components (chart and time bar):

KeyLines.create([{
    id: 'kl',
    type: 'chart'
}, {
    id: 'tl',
    type: 'timebar'
}], componentLoaded);

We can then refer to each distinct component in the loading function:

var timebar;

function componentLoaded(err, loadedComponent) {
    chart = loadedComponent[0];
    timebar = loadedComponent[1];
    ...
}

We need to set up our time data in the appropriate way, to include timestamps – either in Epoch or JavaScript format. We do this by assigning a date object to the dt property, and value (if needed) to the v property:

var links = obj.links.map(function(link) {
    return {
        ...
        dt: [new Date(link.Event1), new Date(link.Event2), new Date(link.Event3)],
        v: [link.Value1, link.Value2, link.Value3]
        ...
    };
});

And then load our data into the graph and time bar:

timebar.load(chartData, function() {
    timebar.zoom('fit', { animate: false })
});

We also need to bind to a filtering function, so only items present in the user-defined time range are visible, and also apply a tweak layout:

timebar.bind('change', tlChange);

var tlChange = function() {
    chart.filter(timebar.inRange, {
        animate: false,
        type: 'link'
    }, function() {
        chart.layout('tweak', {
            animate: true,
            time: 400
        });
    });
};

Finally, we can add some interaction to display a selection line for the selected nodes and its neighbors using timebar.selection():

...
tlSelect(chart.selection());
...

function tlSelect(selection) {
    var selectionObject = [{
        id: selection,
        index: 0,
        c: '#79b300'
    }];
    timebar.selection(selectionObject);
}

With just these few lines of code, we can produce a truly impressive dynamic graph visualization (watch the screencast in the next section).

Dynamic network with D3

D3 has some helpful functions to manage time based data. D3.time.scale, for instance, helps calculating responsive time axis and provide ticks based on time intervals. It is easy then to position elements at the correct time-point, and with D3 .filter method you can  shows items in a specific time range.

Apart from that, you’re on your own. Creating a time bar, especially one that synchronizes with a graph component, is a hugely complex task. Far too complex for this blog post.

Geospatial networks

Another often crucial dimension to data is geography. If nodes have geolocations, users will want to see their network’s geo trends.

Geospatial networks with KeyLines

The good news is that with a ridiculously small amount of code we can enable the map mode in KeyLines!

We implement chart.map().show() and chart.map().hide() functions, binding them to two different html buttons, and push coordinates in the pos property. If you remember, our data was about fake US companies, and we had latitude and longitude already stored for each one of them.

...
pos: {
    lat: node.Latitude,
    lng: node.Longitude
},
...

$('#mapOn').click(function() {
    chart.map().show();
});

$('#mapOff').click(function() {
    chart.map().hide();
});

Since KeyLines used Leaflet library to draw maps, we also add a link to Leaflet assets in our html file.

<script src="map/leaflet.js" type="text/javascript"></script>

This way, we can easily get our graph, timeline and map working together. The result is again impressive and some further customization is possible using map options and functions.

Geospatial networks with D3

D3 has native support for map projections and geometric shapes. You can also implement a 3rd party map drawing library such as Leaflet, as KeyLines does. But again, no built-in function will take care of switching between network and map modes, and you need to code this by yourself.

If you can get all of this working alongside a time bar too, well… get in touch. We’re always looking for outstanding JavaScript developers.

A quick note on performance

When talking about performance in web based graph visualization, besides code optimization, discussions often focus on rendering engines: SVG, Canvas, or WebGL?

KeyLines uses HTML5 Canvas as a default renderer, this can handle many more elements than SVG, the default of D3. KeyLines also provides a WebGL renderer which can leverage the GPU calculating power to boost performance even more.

It is possible to use Canvas with D3, but you can’t use D3 built-in drawing functions and you need to code your own completely from scratch. Switching between rendering modes in KeyLines is as easy as changing an option.

The result: KeyLines is by far the better option for visualizing very large graphs, or even moderately large graphs.

Conclusion – manageable complexity

At the end of this post series I can confirm my previous conclusions:

• As a graph-specific library, KeyLines provides a better graph visualization experience for users and developers alike.
• The KeyLines API is beautifully designed, and developing with it is simpler and faster.
• D3 provides great flexibility, but the trade-off is time, resource and lack of boundaries. Put simply, it’s easier to build a terrible graph visualization in D3, and easy to build something powerful and beautiful with Keylines.

Building graph visualizations is always fun, regardless of the tool you choose and whatever data you’re looking at. I would encourage you to try both and see how you get on.

Try KeyLines for yourself

If you want to try KeyLines for yourself and see how easy and fast it is to visualize your connected data, you can request a free trial, using the form at the bottom of this page.

As always, we love to hear your questions, suggestions and opinions, so please let us know what you think!

The post Building advanced graph visualizations with D3 and KeyLines appeared first on Cambridge Intelligence.