A new clustering method based on multipartite networks

The multipartite graph

A multipartite graph (Van Dam, Koolen & Tanaka, 2016) is a graph whose nodes can be divided into disjoint sets, which are also called layers, within which no two nodes are adjacent. A d-partite graph contains d layers.

MN-C uses a multigraph denoted by G(V, E|X), where V is the set of nodes and E is the set of edges constructed from the data set X. The number of layers of G is equal to the number of attributes of X, denoted by d in this article. Each layer has a maximum of k nodes, where k is the number of classes that we are searching for.

 
_______________________ 
Algorithm 1 Construction of Multipartite Network___________________________________ 
  1:  input:  data set X, number of classes k; 
  2:  output:  network G = (V,E|X); 
  3:  V = ∅; 
  4:  for each attribute j do 
 5:     Compute k intervals Ijl, l = 1,...,k {of equal size} between min(Xj) and 
        max(Xj); 
  6:     Add node vjl to V , where j is the layer of the node 
  7:  end for 
 8:  for each data instance xi ∈ X do 
 9:     Create path v1l1,v2l2,...,vdld  with xij ∈ Ijlj  ∀j=1,...,d − 1, by adding 
        edges (vjlj,vj+1,lj+1) to E; if an edge already exists, increase its weight 
        by 1; 
10:  end for 
11:  Remove empty nodes from V ; 
12:  return d-partite graph G = (V,E|X), and I = {Ijl}j=____1,d,l∈{1,...,k} intervals 
     corresponding to nodes;___________________________________________________________________    

Initial clusters

Each instance of the data set represents a network path from the first layer to the last one. It is reasonable to assume that instances representing the same path may belong to the same cluster. In the first step of MN-C, instances that share a common path throughout all the layers of the network are placed in the same cluster.

Depending on the structure of the data, this procedure may result in any number of clusters, which may be greater than k, due to the number of possible paths in the network. Not all paths in the network represent instances. We denote by k₀ the number of clusters resulting from the this initialization of the clustering. In what follows, a procedure to merge clusters in order to reduce their number is performed, if necessary, i.e., if k₀ > k.

Merging clusters

A cluster C represents a path starting from the first layer to the last layer of the network G. In the initial stage, each cluster contains one path. Clusters are merged by finding paths that have the most common elements. Merging is performed in the order of the number of instances corresponding to each cluster, starting with the smallest ones. The size of the cluster is denoted by μ(C) and is the number of instances assigned to it. In the initial stage, it represents the number of instances that belong to all node intervals from the cluster.

It is assumed that larger clusters are more stable, while those containing fewer instances should be merged. In order to find, for each cluster C, the cluster for it to be merged with, we compute the following strength indicator for two clusters C and C′: (1)${\displaystyle S\left(C,C^{\prime }\right)=\sum _{e_{i,i+1}\in C,C^{\prime }}{w}_{ij}.}$ Thus, S adds the weights of common edges between two clusters. Each cluster is merged with the one for which the strength indicator is the highest, i.e., they have more edges/instances in common. Thus (2)${\displaystyle C^{\ast }=\underset{C^{\prime }}{argmax}S\left(C,C^{\prime }\right).}$ Clusters C and C^∗ are merged by placing all instances from C in C^∗. All clusters are merged except the k largest ones. This process reduces the number of clusters from k₀ to k₁. To reach the desired number of clusters k, the process is repeated for several iterations while the number of clusters is greater than k, and stops when it reaches it, or gets smaller than k.

Outcome

The outcome of MN-C is a set of clusters, corresponding to network paths. Each instance in the data is assigned to a cluster.

 
_______________________________________________________________________________________________________ 
Algorithm 2 MN-C algorithm______________________________________________________________ 
  1:  Input: Data set X ∈ Rn×d, number of clusters k 
 2:  Output: Set of clusters C1,C2,...,Ck; 
  3:  Construct weighted multipartite network G = (V,E) (Algorithm 1) 
  4:  Assign all data instances belonging to the same network path from the first 
     to last layer to the same cluster forming k0 clusters; 
  5:  it = 0; 
  6:  Change = True; 
  7:  while kit > k and Change do 
 8:     Order clusters in ascending order of size μ(C); 
  9:     for each cluster Cl,l=1,...,kit − k do 
10:         Set C∗l = argmaxz=l+1,kitS(Cl,Cz); 
11:         if S(Cl,C∗l) ⁄= 0 then 
12:            Set Cl = C∗l; 
13:         else 
14:            Set C∗l = argmaxz=l+1,kit|ν(Cl,Cz)|; 
15:            if |C∗l|⁄= 0 then 
16:                Set Cl = C∗l; 
17:            end if 
18:         end if 
19:     end for 
20:     If no change has been made to clusters, Change = False 
21:     it ← it + 1; 
22:  end while 
23:  return  Clusters C1,C2,...,Ckit−1__________________________________________________________________________    

Synthetic data sets

In order to illustrate the behaviour of MN-C on different types of data, a set of synthetic data sets with various characteristics was generated by using the make_classification (https://scikit-learn.org/stable/modules/generated/sklearn.datasets.make_classification.html, accessed Jan 2023) function in Python, by combining the following parameters:

• number of instances: 100, 200, 500, 1000, 2000;

• number of attributes: 30, 50, 100, 150, 200, 250, 1000;

• class separator: 0.1, 0.5, 1, controlling the overlap of different clusters;

• number of classes: 30, 50, 100, 150, 200, 250, 1000.

All reasonable combinations of parameter were considered, i.e., excluding the settings with more classes than the number of instances.

Real-world data sets

A selection of data sets used for clustering and classification from the UCI Machine learning repository (Dua & Graff, 2017) is presented. The names and characteristics of the data are listed in Table 1.

Comparisons with other methods

MN-C is compared with 4 clustering methods that aim to find a given number of clusters: Kmeans, Gaussian Mixture (GM), Affinity Propagation (AP), Birch, and Zaki & Meira Jr. (2014); Frey & Dueck (2007); Zhang, Ramakrishnan & Livny (1996). Their corresponding implementation in the sklearn Python library (Pedregosa et al., 2011) is used with its default parameters. In addition, for the real-data sets, the results are also compared to two spectral clustering methods, the standard implementation in Python, which we will call SC and the Ultra-Scalable Spectral Clustering (UNSPEC), using the code provided by the authors (Huang et al., 2020).

Performance evaluation

In order to evaluate and compare the results, the normalized mutual information indicator (NMI) is used (Zaki & Meira Jr., 2014). NMI takes values between 0 and 1 and can be used to compare results provided by a clustering method with a baseline represented by the known clusters. Values closer to 1 indicate a better match.

Synthetic data sets

We generated 588 synthetic data sets. The results are summarised in Fig. 2 and Table 2. Figure 2 presents scatter plots of the values of the NMI obtained by MN-C and each of the other methods. It is used to illustrate the overall performance of MN-C compared to the other method: points located above the first bisector (represented in each image by a grey dashed line) indicate that MN-C performs better. The points located below it indicate that the NMI obtained by the other method are better. For each method, the plot is separated based on the values of the class separator parameter, as this is the one that controls the difficulty of the clustering problem.

Table 2 summarises the results obtained on the synthetic data sets in the following manner: for each characteristic of the data set, the percentage of results for which MN-C obtained higher NMI than the other method is presented. Next to each number, an asterisk (*) indicates if, considering all data sets with the same characteristic, a t-test comparing MN-C’s values of the NMI finds them to be significantly greater than those obtained by the other method. A minus sign (-) indicates that overall there is no significant difference between the methods. An (x) indicates that the results obtained by the other method are significantly better.

The results indicate that MN-C is a competitive method, providing better results than Kmeans, Gaussian Mixture, and Affinity propagation, but not better than the Birch method, for the synthetic data sets. However, Fig. 2 shows that while the results obtained by MN-C are worse in most cases, they are close to those obtained by Birch as they align to the first bisector. We find in general MN-C better than the other three methods in more instances for the most difficult settings, for example for class separator values of 0.1, a smaller number of instances, a large number of attributes, and a larger number of classes.

Varying the number of clusters

MN-C uses the number of clusters as a parameter. For different values of this parameter, it will divide the data into clusters that also contain grouped data, in a manner similar to that of the other clustering methods. For example, given a synthetic data set with 100 instances, 150 attributes, 30 classes, and a class separation parameter of 0.1, the NMI obtained by all the methods, except AP which determines the number of clusters on its own, have a similar trend, as illustrated in Fig. 3. The increasing values of NMI for all the methods indicate that when more clusters are obtained, instances that belong to the same clusters are grouped together in sub-clusters. Since NMI is an external performance measure it cannot be used to detect the number of clusters, but it can show that the performance of the method may be considered robust with respect to this parameter, as it places instances from the same cluster together. It may also indicate that any method of determining the number of clusters in the data based on some internal quality measure, such as the elbow method (Yager & Filev, 1994; Thorndike, 1953), or information-based methods (Sugar & James, 2003), can be used with MN-C in a manner similar to how they are used with other clustering methods (Fu & Perry, 2020). Figure 4 illustrates the values of three internal indicators that are known to be used with an elbow method to evaluate the number of clusters based on the results of a learner for the same data set as in Fig. 3. The silhouette score, distortion, and inertia all have descending values; a linear trend may be observed at their left.

Real-world benchmarks

Results obtained by the four methods on the real-world benchmarks are presented in Table 4. The value of the NMI and number of clusters determined by each method are indicated. The data sets vary in the number of clusters, instances, and attributes. We show a variety of situations in which the results obtained by MN-C are better than those obtained by the other methods (including Birch, unlike the case of the synthetic data sets). The results also illustrate the reality of the variability in clustering performance with large differences between the methods for some data sets, e.g., R6, for which MN-C obtains a value of 0.662 for the NMI, while the other methods obtain 0.041, 0.026, 0.453, and 0.031, respectively. AP obtains 0.453 but with 11 clusters instead of two. This situation arises also for other data sets. While these results cannot be generalized, they do indicate the potential of MN-C to undercover the clustering structure of different real-world data sets.