LinkPred: a high performance library for link prediction in complex networks

The network data structures

The life cycle of a network has two distinct phases. In the pre-assembly phase, it is possible to add nodes and edges to the network. It is also possible to access nodes and translate external labels to internal IDs and vice versa. However, most functionalities related to accessing edges are not yet available. As a result, the network at this stage is practically unusable. To be able to use the network, it is necessary to assemble it first. Once assembled, no new nodes or edges can be added (or removed) to the network. The network is then fully functional and can be passed as an argument to any method that requires so.

To build a network, an empty network is first created by calling the default constructor:

Most classes in LinkPred manipulate networks through smart pointers for efficient memory management. To create a shared pointer to a UNetwork object:

Notice that the class UNetwork is a class template, which is instantiated with the default template arguments. In this default setting, the labels are of type std::string, whereas internal IDs are of type unsigned int, but UNetwork can be instantiated with several other data types if wanted. For instance, the labels can be of type unsigned int, which may reduce storage size in some situations.

Adding nodes is achieved by calling the method addNode, which takes as parameter the node label and returns an std::pair containing, respectively, the node ID and a Boolean which is set to true if the node is newly inserted, false if the node already exists. The nodes IDs are guaranteed to be contiguous in 0, …, n − 1, where n is the number of nodes.

The method addEdge is used to create an edge between two nodes specified by their IDs (not their labels):

The last step in building the network is to assemble it:

The method assemble initializes the internal data structures and makes the network ready to be used.

Nodes can be accessed through iterators provided by nodesBegin() and nodesEnd(). For convenience, the iterator points to a pair, the first element of which is the internal ID, whereas the second is the external label.

Alternatively, one can iterate over labels in a similar way using the iterators labelsBegin() and labelsEnd():

It is also possible to translate labels to IDs and vice versa using getID(label) and getLabel(id) respectively. Oftentimes, one would want to iterate over a random sample of nodes instead of the whole set. This can be easily done using the two methods rndNodesBegin and rndNodesEnd.

Information on edges can only be accessed after assembling the network. One way to access edges is to iterate over all edges in the network. This can be done using the method edgesBegin() and edgesEnd(). As it is the case with nodes, it is possible to access a random sample of edges using rndEdgesBegin and rndEdgesEnd. LinkPred offers the possibility to iterate over negative links in the same way one iterates over positive edges. This can be done using the method nonEdgesBegin() and nonEdgesEnd():

It is also possible to iterate over a randomly selected sample of negative links using rndNonEdgesBegin and rndNonEdgesEnd.

To represent directed networks, LinkPred offers the class DNetwork, which offers a very similar interface to UNetwork.

Maps

Maps are a useful way to associate data with nodes and edges. Two types of maps are available in LinkPred: node maps (class NodeMap) and edge maps (class EdgeMap), both member of UNetwork. The first assigns data to the network nodes, whereas the latter maps data to edges (see Fig. 2 for an example).

Creating a node map is achieved by calling the method createNodeMap on the network object. This is a template method with the mapped data type as the only template argument. For example, to create a node map with data type double over the network net:

Creating an edge map can be done in a similar way:

Both NodeMap and EdgeMap offer the same interface, which in fact is similar to std::map. This includes the operator [], the methods at, begin, end, cbegin and cend. From the performance point of view, NodeMap offers constant time access to mapped values, whereas EdgeMap requires logarithmic time access (O(logm), m being the number of edges).

If a node map is sparse, that is, has non-default values only on a small subset of the elements, it is better to use a sparse node map. To create a sparse node map:

Notice that the method takes as input one parameter that specifies the map’s default value (in this case, it is 0.0). Hence, any node which is not explicitly assigned a value is assumed to have the default value 0.0.

Graph traversal and shortest paths algorithms

LinkPred provides two classes for graph traversal: BFS, for Breadth-First traversal, and DFS for Depth-First traversal. They both inherit from the abstract class GraphTraversal, which declares one virtual method traverse. It takes as parameter the source node from where the traversal starts and a reference to a NodeProcessor object, which is in charge of processing nodes sequentially as they are visited.

In addition to graph traversal routines, LinkPred contains an implementation of Dijkstra’s algorithm for solving the shortest path problem. To use it, it is first necessary to define a length (or weight) map that specifies the length associated with every edge in the graph. A length map is simply a map over the set of edges, that is, an object of type EdgeMap which can take integer or double values. The class Dijkstra offers two methods for computing distances:

• The method getShortestPath, which computes and returns the shortest path between two nodes and its length.

• The method getDist, which returns the distance between a source node and all other nodes. The returned value is a node map, where each node is mapped to a pair containing the distance from the source node and the number of edges in the corresponding shortest path.

Both methods run Dijkstra’s algorithm, except that getShortestPath stops once the destination node is reached, whereas getDist continues until all reachable nodes are visited.

Computing shortest-path distances in large networks requires not only considerable time but also significant space resources. Consequently, efficient management of memory is necessary to render the task feasible in such situations. The abstract class NetDistCalculator provides an interface for an additional layer over the class Dijkstra which facilitates its use and can serve to manage memory usage. A NetDistCalculator object is associated with a single length map and provides two methods for computing distances:

• getDist(i, j): Computes and returns the distance between the two nodes i and j. The returned value is an std::pair, with the first element being the distance, whereas the second is the number of hops in the shortest path joining the two nodes.

• getDist(i): Computes and returns a node map containing the distances from node i to all other nodes in the network.

LinkPred has two implementations of NetDistCalculator: ESPDistCalculator, an exact shortest path distance calculator which caches distances according to different strategies to balance memory usage and computation, and ASPDistCalculator, an approximate shortest path distance calculator. The approximation used in ASPDistCalculator works as follows. A set $\mathcal{L}$ of nodes called landmarks is selected, and the distance from each landmark to all other nodes is pre-computed and stored in memory. The distance between any two nodes i, j is then approximated by: (1)${\displaystyle d_{ij}\simeq \underset{k\in \mathcal{L}}{min}\left[d_{ik}+d_{kj}\right].}$ The landmarks are passed to ASPDistCalculator object using the method setLandmarks. Naturally, by increasing the number of landmarks, more precision can be obtained, be it though at a higher computational and memory cost.

Graph embedding algorithms

Graph embedding consists in transforming the graph’s nodes and edges into elements of a low-dimensional vector space while preserving, as much as possible, its structural properties (Goyal & Ferrara, 2018c). It is a problem with important applications in various fields, including link prediction (Goyal & Ferrara, 2018c; Kazemi & Poole, 2018; Alharbi, Benhidour & Kerrache, 2016; Wang, Cui & Zhu, 2016), product recommendation (Koren, Bell & Volinsky, 2009), data visualization (van der Maaten & Hinton, 2008; Tang et al., 2016; Cao, Lu & Xu, 2016), and node classification (Bhagat, Cormode & Muthukrishnan, 2011; Tang, Aggarwal & Liu, 2016).

LinkPred contains several state-of-the-art graph embedding algorithms, some of which are implemented from scratch, whereas others are based on publicly available implementations. These include methods based on matrix decomposition, namely Locally Linear Embedding (Roweis & Saul, 2000) implemented in the class LLE, Laplacian Eigenmaps (Belkin & Niyogi, 2001) implemented in the class LEM, and Matrix Factorization (Koren, Bell & Volinsky, 2009) (also referred to as Graph Factorization in Goyal & Ferrara, (2018c); Ahmed et al., (2013)) implemented in the class MatFact. Also available are methods based on random walks, including DeepWalk (Perozzi, Al-Rfou & Skiena, 2014) implemented in the class DeepWalk, Large Information Networks Embedding (LINE) (Tang et al., 2015), implemented in the class LINE, LargeVis (Tang et al., 2016) implemented in the class LargeVis, and node2vec (Grover & Leskovec, 2016), which is implemented in the class Node2Vec. Additionally, the librray includes the implementation of the Hidden the Metric Space Model (HMSM) embedding method (Alharbi, Benhidour & Kerrache, 2016) available through the class HMSM.

To provide a uniform interface, all embedding algorithms implemented in LinkPred inherit from the abstract class Encoder, which declares the following methods:

• The method init, which is first called to initialize the internal data structures of the encoder. This is a pure virtual method of the class Encoder and must be implemented by derived classes.

• Once the encoder is initialized, the method encode, also a pure virtual method, is called to perform the embedding. This step typically involves solving an optimization problem, which can be computationally intensive both in terms of memory and CPU usage, especially for very large networks. The dimension of the embedding space can be queried and set using getDim and setDim respectively.

• The node embedding or the node code, which is the vector of coordinates assigned to the node, can be obtained by calling the method getNodeCode. The edge code is by default the concatenation of its two nodes’ codes and can be obtained using getEdgeCode. Hence, in the default case, the edge code dimension is double that of a node. Classes that implement the Encoder interface may change this default behavior if desired. The user can query the dimension of the edge code using the method getEdgeCodeDim.

Having a unified interface for encoders allows embedding algorithms to be easily combined with different classifiers and similarity measures to obtain various link prediction methods, as explained in the next sections. It also allows users to use their own embedding algorithms to build and test new link prediction methods.

The predictor interface

As stated above, all link predictors for undirected networks must inherit from the abstract class ULPredictor. It declares three important pure virtual methods that the derivative classes must implement:

• The method void init(): This method is used to initialize the predictor’s state, including any internal data structures.

• The method void learn(): In algorithms that require learning, it is in this method that the model is built. The learning is separated from prediction because, typically, the model is independent of the set of edges to be predicted.

• The method double score(Edge const & e): returns the score of the edge e (usually a non-existing edge).

In addition to these three basic methods, ULPredictor declares the following three virtual methods, which by default use the method score to assign scores to edges, but which can be redefined by derived classes to achieve better performance:

• The method void predict(EdgeRndIt begin, EdgeRndIt end, ScoreRndIt scores): In this method, the edges to be predicted are passed to the predictor in the form of a range (begin, end) in addition to a third parameter (scores) to which the scores are written. This is a virtual method that uses the method score to assign scores to edges and can be redefined by derived classes to provide better performance.

• The method std::pair¡NonEdgeIt, NonEdgeIt¿ predictNeg(ScoreRndIt scores) predicts the score for all negative (non-existing) links in the network. The scores are written into the random output iterator scores. The method returns a pair of iterators begin and end to the range of non-existing links predicted by the method.

• The method std::size_t top(std::size_t k, EdgeRndOutIt eit, ScoreRndIt sit) finds the k negative edges with the top scores. The edges are written to the output iterator eit, whereas the scores are written to sit.

The class ULPredictor offers default implementations for the methods top, predict and predictNeg. Sub-classes may use these implementations or redefine them to achieve better performance.

The abstract class DLPredictor plays the same role as ULPredictor but for link predictors in directed networks. It offers the same interface as the latter but with different default template arguments and methods implementation.

Receiver operating characteristic curve (ROC)

One of the most important performance measure used in the field of link prediction is the receiver operating (ROC) curve, in which the true positive rate (recall) is plotted against the false positive rate. The ROC curve can be computed using the class ROC. Figure 4A shows an example ROC curve obtained using this class.

The default behavior of the ROC performance measure is to compute the positive and negative edge scores and then compute the area under the curve, which may lead to memory issues with large networks. To compute the area under the curve without storing both types of scores, the class ROC offers a method that streams scores without storing them. To enable this method, call setStrmEnabled(bool) on the ROC object. To specify which scores to stream use the method setStrmNeg(bool). By default, the negative scores are streamed, while the positive scores are stored. Passing false to setStrmNeg switches this. In addition to consuming little memory, the streaming method supports distributed processing (in addition to shared memory parallelism), making it suitable for large networks.

Precision–recall curve

The precision–recall (PR) curve is also a widely used measure of link prediction algorithms’ performance. In this curve, the precision is plotted as a function of the recall. The PR curve can be computed using the class PR. The area under the PR curve can be computed using two integration methods:

• The trapezoidal rule which assumes a linear interpolation between the PR points.

• Nonlinear interpolation as proposed by Jesse Davis and Mark Goadrich (Davis & Goadrich, 2006).

The second method is more accurate, as linear integration tends to overestimate the area under the curve (Davis & Goadrich, 2006). Furthermore, the implementation of Davis-Goadrich nonlinear interpolation in LinkPred ensures little to no additional cost compared to the trapezoidal method. Figure 4B shows an example PR curve obtained using the class PR.

General performance curves

LinkPred offers the possibility of calculating general performance curves using the class GCurve. A performance curve is, in general, defined by giving the x and y coordinates functions. These are passed as parameters, in the form of lambdas, to the constructor of the class GCurve. The associated performance value is the area under the curve computed using the trapezoidal rule (linear interpolation). For example, the ROC curve can be defined as:

The two first parameters of the constructors are lambdas having the signature:

Top precision

The top precision measure is defined as the ratio of true positives within the top l scored edges, l > 0 being a parameter of the measure (usually l is set to the number of links removed from the network). Top precision is implemented by the class TPR, and since it is not a curve measure, this class inherits directly from PerfMeasure. The class TPR offers two approaches for computing top-precision. The first approach requires computing the score of all negative links, whereas the second approach calls the method top of the predictor. The first approach is, in general, more precise but may require more memory and time. Consequently, the second approach is the performance measure of choice for very large networks.