HSMVS: heuristic search for minimum vertex separator on massive graphs

Theoretical algorithms

Because MVS has proven to be NP-hard (Bui & Jones, 1992; Fukuyama, 2006), it seems impossible to design exact algorithms with the complexity of polynomial time. Thus, most theoretical works on MVS focused on designing approximation algorithms, which aims at improving the approximation ratio for this NP-hard combinatorial optimization problem. Leighton & Rao (1999) presented an approximation algorithm for MVS, which is based on linear programming, and the algorithm gives an approximation ratio of O(logn) for MVS. Then, Feige, Hajiaghayi & Lee (2008) developed an approximation algorithm for MVS, which is based on novel linear and semidefinite program relaxations, and obtained the approximation ratio of $O\left(\text{log}\sqrt{opt}\right)$, where opt is the size of an optimal vertex separator.

Practical algorithms

Even though a number of great contributions have been made on the theoretical analysis of MVS solving, the performance of theoretical algorithms for MVS is still unsatisfactory in practice. As MVS has important applications in real-world situations, such as VLSI design, computational biology, etc (Balas & de Souza, 2005; Biha & Meurs, 2011; Benlic & Hao, 2013; Zhang & Shao, 2015; Dagdeviren, Akram & Farzan, 2019; Furini et al., 2022), a number of practical algorithms for MVS have been proposed.

As mentioned in the ‘Introduction’, practical algorithms for MVS can be classified into two categories: exact algorithms and heuristic search algorithms. Exact algorithms are guaranteed to prove optimal solutions, but they may fail to return good-quality solutions within reasonable time on solving large-sized instances (Benlic & Hao, 2013). Heuristic search algorithms could not prove optimality for the solutions they find, but they are able to seek out good-quality solutions for large-sized instances efficiently.

Most previous works on practical MVS solving focus on designing and improving exact algorithms. In de Souza & Balas (2005), developed a branch-and-cut algorithm for MVS, which is based on the mixed integer programming formulation (Balas & de Souza, 2005). After that, Biha & Meurs (2011) designed an exact algorithm for MVS, on the basis of new classes of valid inequalities for the associated polyhedron. Further, de Souza & Cavalcante (2011) proposed a hybrid algorithm, which is built on a Lagrangian relaxation framework. Recently, Althoby, Biha & Sesboüé (2020) introduced a practical method which combines branch-and-bound procedure, linear programming technique and greedy algorithm.

In the context of MVS solving by heuristic search, Benlic & Hao (2013) developed the first local search algorithm called BLS for solving MVS. In order to improve the performance, BLS incorporates several sophisticated heuristics (including a greedy hill-climbing component, an adaptive perturbation mechanism, a hashing function and a jumping-magnitude determining component), which introduce six instance-dependent parameters. There exists an improved version of BLS, which is called BLS-RLE (Benlic, Epitropakis & Burke, 2017). BLS-RLE introduces an effective parameter control mechanism that draws upon ideas from reinforcement learning theory to reach an interdependent decision. According to the computational results reported in the literature (Benlic & Hao, 2013; Benlic, Epitropakis & Burke, 2017), BLS is able to handle graphs with up to 3,000 vertices and runs much faster than a number of high-performance exact algorithms and BLS-RLE exhibits its effectiveness in solving the MVS problem. Besides, Zhang et al. (2015) proposed an improved K-OPT local search algorithm named New_K-OPT. The experimental results reported in the literature (Zhang et al., 2015) show that New_K-OPT exhibits relatively better performance compared to variable neighborhood search, simulated annealing and Relax-and-Cut (de Souza & Cavalcante, 2011) on a number of graphs.

The construction procedure

The extending stage: In the beginning, three vertex sets A, B and C are initialized as ∅ (line 1). Then, for each vertex v ∈ V, the function puts v into one of these three sets according to the following rules.

• With probability p, if |A| < b, the function puts v into A; if |A| ≥ b and |B| < b, the function puts v into B; if |A| ≥ b and |B| ≥ b, the function puts v into C (lines 3–6).

• Otherwise (with probability 1 − p), if |B| < b, the function puts v into B; if |B| ≥ b and |A| < b, the function puts v into A; if |B| ≥ b and |A| ≥ b, the function puts v into C (lines 7–11).

The fixing stage: According to the rules in the extending stage, there might be a number of edges that connect some vertices in set A and their neighboring vertices in set B, which makes the resulting solution {A, B, C} infeasible. To construct a feasible solution, the function tries to move some vertices from set B to set C. For each vertex v ∈ B, the function checks whether there are neighboring vertices of v in set A; if this is the case, the function moves vertex v into set C in order to resolve the contradiction (lines 13–15). Finally, the function returns s = {A, B, C} as the solution.

As stated in the literature (Cai, 2015), it is important to design low time-complexity function to generate the initial solution for massive graphs, because high time-complexity construction procedure would make the algorithms inefficient. According to the pseudo-code in Algorithm 2 , we could easily derive the following lemma (Lemma 1).

In fact, most real-world massive graphs are sparse ones (Barabási & Albert, 1999; Eubank et al., 2004; Cai, 2015). Thus, in most cases, the complexity of our function Construct_Solution is usually lower than O(|V|²), which indicates that our construction procedure is practical for a large number of real-world massive graphs.

The modification heuristic

The modification heuristic also plays a critical role in the HSMVS algorithm. An important issue of designing an effective modification heuristic is to strike a good balance between intensification and diversification (Li & Huang, 2005). Inspired by the success of two-mode heuristic search algorithms in Boolean satisfiability solving (Balint & Fröhlich, 2010; Li & Li, 2012; Cai & Su, 2013; Luo, Su & Cai, 2014), we propose an effective two-mode modification heuristic named Modify_Solution in the context of MVS solving.

Essentially, the heuristic Modify_Solution modifies the current solution by moving a vertex v from set C to the target set X ∈ {A, B} and resolving the contradictions by moving to set C those vertices, which are v’s neighbors and are currently in the opposite set Y (Y = {A, B}∖X). Clearly, the most important issue of the heuristic is to decide the moving vertex v and the target set X.

Before describing the details of Modify_Solution, we introduce the basic operation in the heuristic. The operation move(v, X, Y), where v ∈ C, X ∈ {A, B}, Y = {A, B}∖X, works as follows. It first moves vertex v from set C to set X. Then, for each vertex w ∈ Y, it checks whether w ∈ N(v); if this is the case, it moves w from set Y to set C to keep the solution legal. We also introduce two evaluating properties score_A and score_B, which are important metrics for evaluating the priority of vertices in set C. The formal definitions of score_A and score_B are given as follows (Definitions 1 and 2).

Given a vertex v ∈ C, the evaluation properties score_A and score_B represent the benefit through performing the operations move (v, A, B) and move (v, B, A), respectively. Also, performing an operation with larger value of score_A or score_B would reduce the value of cost to the largest extent. Therefore, it is advisable to select and conduct an operation with large value of score_A or score_B.

These properties play important roles in the reconstruction of solutions and reduction of the cost.

We present the pseudo-code of the whole heuristic Modify_Solution in Algorithm 3 , and describe it in detail. Our heuristic Modify_Solution switches between two modes, i.e., the random mode and the greedy mode, in order to strike a good balance between intensification and diversification. The function Modify_Solution activates which mode depending on a probability wp. With the probability wp, Modify_Solution works in the random mode (lines 1–5); otherwise (with the probability 1 − wp), Modify_Solution works in the greedy mode (lines 6–19). The procedures of the random mode and the greedy mode are described as follows.

The random mode: In this mode, the heuristic employs the random walk component to strengthen diversification. The random walk component first randomly selects a vertex v from set C, and then randomly picks a target set X from {A, B}. If set X is set A, then the heuristic performs the operation move(v, A, B); otherwise (set X is set B), the heuristic executes the operation move(v, B, A).

The greedy mode: In this mode, the heuristic applies the approximate best selection component to contribute to intensification, inspired by the success of Best from Multiple Selections (BMS) in the context of minimum vertex cover (Cai, 2015). The approximate best selection component first chooses t vertices from set C, and among these t vertices selects the vertex with the greatest score_A, denoted as v_A (lines 7–11). Then, the heuristic also chooses t vertices from set C, and among these t vertices selects the vertex with the greatest score_B, denoted as v_B (lines 12–16). Finally, the heuristic checks whether score_A(v_A) is greater than score_B(v_B); if this is the case, it executes the operation move(v_A, A, B); otherwise (score_A(v_A) is not greater than score_B(v_B)), it executes the operation move(v_B, B, A).

Finally, our heuristic Modify_Solution denotes the resulting sets A, B, C as sets A′, B′ and C′, respectively, and then returns s′ = {A′, B′, C′} as the resulting solution.

Remark: We note that the solution s′ returned by the heuristic Modify_Solution might be infeasible. If this is the case, the algorithm would first rollback the resulting solution s′ = {A′, B′, C′} to s = {A, B, C}, and then randomly moves a vertex from set A to set C (or moves a vertex from set B to set C).

The benchmarks

We evaluate HSMVS on all 139 graphs collected in a public and standard graph I benchmark (https://lcs.ios.ac.cn/ caisw/graphs.html), which is originally collected from Network Repository (Rossi & Ahmed, 2015a; Rossi & Ahmed, 2015b) and consists of a broad range of real-world massive simple undirected graphs. Most of these graphs are encoded from real-world applications. In practice, these real-world massive graphs have been utilized in testing practical algorithms for well-known NP-hard combinatorial optimization problems in graph theory, including minimum vertex cover (Luo et al., 2019; Li et al., 2020), minimum dominating set (Chen et al., 2023), maximum clique (Rossi et al., 2014) and graph coloring (Rossi & Ahmed, 2014).

The graphs tested in our experiments contain a variety of real-world networks, and can be classified into 12 categories, including biological networks, collaboration networks, Facebook networks, infrastructure network, interaction networks, recommendation networks, Retweet networks, scientific computing, social networks, technological networks, temporal reachability networks and web graphs. For these graphs evaluated in our experiments, all the vertices are given unit weights, and $b=\lfloor \frac{1.05\left|V\right|}{2}\rfloor$ recalling that b can be regarded as the limitation size and firstly introduced in Section ‘Preliminaries’. These benchmarking settings are suggested by the literature (Benlic & Hao, 2013).

The competitors

Our HSMVS algorithm is compared against an effective breakout local search solver BLS (Benlic & Hao, 2013), an improved K-OPT local search solver New_K-OPT (Zhang & Shao, 2015), and an optimized version of BLS, called BLS-RLE (Benlic, Epitropakis & Burke, 2017).

• The BLS solver (Benlic & Hao, 2013) is the first local search solver for solving the MVS problem, and it achieves effectiveness in solving MVS instances. According to the experiments in the literature (Benlic & Hao, 2013), BLS performs significantly better than a number of high-performance exact solvers (de Souza & Balas, 2005; Biha & Meurs, 2011). As reported in the literature (Hager & Hungerford, 2015), BLS exhibits its effectiveness in solving MVS on random graphs.

• The New_K-OPT solver (Zhang & Shao, 2015) is a high-performance, improved K-OPT local search solver for solving the MVS problem. The experimental results reported in the literature (Zhang & Shao, 2015) show that New_K-OPT exhibits relatively better performance compared to several methods, such as variable neighborhood search, simulated annealing and Relax-and-Cut (de Souza & Cavalcante, 2011), on a number of graphs.

• The BLS-RLE solver (Benlic, Epitropakis & Burke, 2017) is an enhancement of BLS. The BLS-RLE solver introduces a new parameter control mechanism, which is designed on the basis of the reinforcement learning theory. As claimed by its authors, this new parameter control mechanism could help the BLS-RLE solver better escape from the local optimum situation. According to the experimental results demonstrated in the literature (Benlic, Epitropakis & Burke, 2017), BLS-RLE performs much better than BLS on various types of graphs.

Experimental setup

Our HSMVS algorithm is implemented in the programming language C++. In our experiments, for HSMVS, the parameter p is set to 0.5, as the initialization should be uniformly random; the parameter wp is set to 0.05 and the parameter t is set to 20 according to preliminary experiments. The local search competitor BLS is an open-source solver and can be downloaded online (http://www.info.univ-angers.fr/pub/hao/BLSVSP/Code/BLS_VSP.cpp). The BLS solver is implemented in the programming language C++. For BLS, we adopt the parameter settings which are reported in the literature (Benlic & Hao, 2013). The BLS solver is implemented in the programming language C++. For BLS-RLE, its implementation is publicly available online. (http://www.epitropakis.co.uk/BLS-RLE/) The BLS-RLE is implemented in the programming language C++, and it is evaluated using the configuration settings that are utilized in the literature (Benlic, Epitropakis & Burke, 2017). The source codes of the improved K-OPT local search competitor New_K-OPT is kindly provided by its author. The New_K-OPT solver is implemented in the programming language C++. For New_K-OPT, we adopt the algorithmic settings which are reported in the literature (Zhang & Shao, 2015). In order to make the empirical comparison fair, all these three algorithms HSMVS, BLS, BLS-RLE and New_K-OPT are statically complied by the compiler g++ with the option ‘-O3’.

All the experiments are carried out on a number of workstations equipped with Intel Xeon E7-8830 2.13 GHz CPU, 24MB L3 cache and 1.0TB RAM under the operating system CentOS 7.0.1406. In our experiments, each solver performs 10 runs on each graph. The cutoff time of each run performed by each solver is set to 1,000 s.

For each graph, we report the best solution quality found by each solver among all 10 runs, denoted by ‘best’¹, the average solution quality over all 10 runs, denoted by ‘avg.’, and the average run time of reporting the best solution in each run, denoted by ‘time’. If a solver fails to report solutions on a graph within the cutoff time among all 10 runs, we mark ‘N/A’ for ‘best’, ‘avg.’ and ‘time’ for the related solver on the related graph.

Furthermore, for each solver on each graph class, we report the number of graphs where the solver finds the best solution quality among all competing solvers in the related experiment, denoted by ‘#best’, the number of graphs where the solver finds the best average solution quality among all competing solvers in the related experiment, denoted by ‘#avg.’, and the average time of reporting the best solution in each run, denoted by ‘time’. If a solver fails to report solutions on all graphs in a graph class, we mark ‘N/A’ for ‘time’ for the related solver on the related graph class. The number of graphs in each graph class is indicated in the column ‘#graph’.

This form of demonstrating experimental results is inspired by the rules of well-known SAT competitions (http://www.satcompetition.org/) and MAX-SAT evaluations (http://www.maxsat.udl.cat/).

Experimental results

In this subsection, we first present the experimental results, and then conduct some discussions about the results.

The comparative results of HSMVS and its competitors BLS, BLS-RLE, New_K-OPT on all real-world massive graphs are reported in Tables 1–7, where Table 1 presents the comparative results on the graph classes of biological networks and collaboration networks, Table 2 presents the comparative results on the graph classes of Facebook networks and infrastructure networks, Table 3 presents the comparative results on the graph classes of interaction networks, recommendation networks, Retweet networks and scientific computing, Table 4 presents the comparative results on the graph classes of social networks and technological networks, Table 5 presents the comparative results on the graph class of temporal reachability networks, Table 6 presents the comparative results on the graph class of web graphs, and Table 7 summarizes the comparative results on all massive real-world graphs.

First we focus on the comparison between HSMVS and BLS. According to the results reported in Tables 1–7, among all 12 graph classes, it is apparent that our HSMVS algorithm performs better than BLS on 9 graph classes (i.e., biological networks, collaboration networks, facebook networks, infrastructure networks, retweet networks, scientific computing, social networks, technological networks and web graphs). On the overall performance, according to Table 7, among all 139 real-world massive graphs, our HSMVS algorithm finds the best solution quality for 96 of them, while BLS does that for only 55 of them; HSMVS finds the best average solution quality for 89 of them, while this figure for BLS is only 52.

Then we concentrate on the evaluation between HSMVS and New_K-OPT. According to the results reported in Tables 1–7, it is clear that HSMVS significantly outperforms New_K-OPT on all 12 graph classes. On the overall performance, seen from Table 7, HSMVS gives the best solution quality for 96 of them, while this figure for New_K-OPT is only 3; HSMVS finds the best average solution quality for 89 of them, while this figure for New_K-OPT is only 2.

Finally, we analyze the comparison between HSMVS and BLS-RLE. According to the results reported in Tables 1–7, our HSMVS algorithm performs better than BLS-RLE on 7 graph classes (i.e., biological networks, collaboration networks, infrastructure networks, retweet networks, social networks, technological networks, and web graphs). On the overall performance, according to Table 7, among all 139 real-world massive graphs, our HSMVS algorithm finds the best solution quality for 96 of them, while BLS-RLE does that for only 83 of them; HSMVS finds the best average solution quality for 89 of them, while this figure for BLS is only 78.

Remark: The experimental results on a broad range of real-world massive graphs in Tables 1–7 present that HSMVS generally performs much better than the effective breakout local search competitor BLS, the improved K-OPT local search competitor New_K-OPT, and the enhanced version of BLS named BLS-RLE, on a large number of real-world massive graphs, indicating that HSMVS shows its superiority on solving real-world massive graphs.

Effectiveness of algorithmic components underlying HSMVS

According to the description of the HSMVS algorithm, it is obvious that the approximate best selection component in the greedy mode and the random walk component in the random mode are the key parts. In order to show the effectiveness of these two components, we develop three alternative versions of HSMVS, which are all modified from HSMVS and are described as follows.

• HSMVS_alt1: This version uses the strict best selection component instead of the approximate best selection component. HSMVS_alt1 differs from HSMVS in lines 7–16 in Algorithm 3 : in lines 7–11, HSMVS_alt1 greedily selects the variable with the greatest score_A from set C, denoted v_A; in lines 12–16, HSMVS_alt2 greedily selects the variable with the greatest score_B from set C, denoted as v_B.

• HSMVS_alt2: This version uses the random selection component instead of the approximate best selection component. HSMVS_alt2 also differs from HSMVS in lines 7–16 in Algorithm 3 : in lines 7–11, HSMVS_alt2 randomly selects a variable from set C, denoted as v_A; in lines 12–16, HSMVS_alt2 randomly selects a variable from set C, denoted as v_B. In another word, HSMVS_alt2 could be considered as a specific version of HSMVS with parameter t = 1.

• HSMVS_alt3: This version does not utilize the random walk component, i.e., working without the random mode (deleting lines 1–5 in Algorithm 3 ). In another word, HSMVS_alt3 could be considered as a specific version of HSMVS with parameter wp = 0.

Then, we conduct extensive empirical evaluations to compare HSMVS with its three alternative versions on the all 139 real-world massive graphs. The experimental setup used in this comparison is the same one used in ‘Experiments’. To make the evaluation fair, all these alternative versions are also implemented in C++, and are statically compiled by g++ with the option ‘-O3’. Furthermore, the parameters settings used in these three alternative versions are the same as in HSMVS.

Table 8 reports the related empirical results of comparing the HSMVS algorithm with all its alternative versions (i.e., HSMVS_alt1, HSMVS_alt2 and HSMVS_alt3) on all 139 real-world massive graphs. As can be seen from Table 8, it is clear that HSMVS stands out as the general best solver in this comparison. Particularly, HSMVS performs much better than all its alternative versions in terms of both the best solution quality and the average solution quality. Among 139 total real-world massive graphs, HSMVS finds the best solution quality for 85 of them, while this figure is only 64, 42 and 71 for HSMVS_alt1, HSMVS_alt2 and HSMVS_alt3, respectively; HSMVS finds the best average solution quality for 83 of them, while this figure is only 56, 35 and 67 for HSMVS_alt1, HSMVS_alt2 and HSMVS_alt3, respectively.

Remark: The empirical results presented in Table 8 show that HSMVS generally performs better than all its alternative versions and thus is the general best algorithm on the real-world massive graphs, which confirms the effectiveness of the approximate best component and the random walk component.

Experiment results on different limitation size

In this subsection, we conduct empirical evaluations to assess the performance of HSMVS on different limitation size. In particular, compared to the setting of limitation size ($b=\lfloor \frac{1.05\left|V\right|}{2}\rfloor$) that is adopted in Section ‘Experiments’, here we set the limitation size to b = ⌊0.6|V|⌋. Also, in this subsection we conduct empirical evaluations on 12 selected graphs, where we randomly select a graph from each graph class. Table 9 reports the comparative results of HSMVS and its competitors on 12 selected graphs with b = ⌊0.6|V|⌋, and Table 10 summarizes the overall results on those 12 selected graphs with b = ⌊0.6|V|⌋. As can be observed from Tables 9 and 10, our HSMVS algorithms still performs generally better than its competitors (i.e., BLS, BLS-RLE and New_K-OPT). According to Table 10, HSMVS gives the best solution quality for 9 of the overall selected graphs, while this figure for BLS, New_K-OPT and BLS-RLE is 0, 0 and 3, respectively. Also, HSMVS finds the best average solution quality for 8 of them, while this figure for BLS, New_K-OPT and BLS-RLE is 0, 0 and 4, respectively. In summary, HSMVS achieves generally better performance than its competitors on a different limitation size (i.e., b = ⌊0.6|V|⌋).

Discussion on the advantage of HSMVS

As presented in Tables 1–7, there is no single best algorithm across all classes of graphs. Hence, in this subsection, we aim to discuss the advantage of HSMVS when compared to its competitors. Particularly, we analyze the experimental results and the features of graphs, for identifying the characteristics of graphs which HSMVS exhibits better effectiveness than its competitors. Figure 3 illustrates the relationship between the practical performance of competing algorithms (including HSMVS and its competitors) and the size of graphs (i.e., the number of graphs’ vertices). In Fig. 3, the X-axis depicts ln(|V|), where |V| represents the number of vertices, while the Y-axis presents ln(|avg| + 1), where |avg| denotes the corresponding algorithm’s obtained average solution quality over all 10 runs. It can be observed that our HSMVS algorithm shows competitive performance on graphs with relatively large number of vertices. As discussed in ‘The HSMVS Algorithm’, our HSMVS algorithm strikes a good balance between intensification and diversification. In this way, when handling graphs with relatively large numbers of vertices, compared to its competitors, our HSMVS algorithm is able to explore a broader solution space in a shorter time, resulting in an advantage on larger-scale graphs.