Towards virtual machine scheduling research based on multi-decision AHP method in the cloud computing platform

Model of energy consumption and overhead of virtual machine migration

In a cloud data center, memory, CPU, cooling systems and other devices all have significant energy demands. Energy consumption of the CPU accounts for approximately 61% of the total power consumed in the data center (Mejahed & Elshrkawey, 2022), so the energy consumption and power consumption of the hosts in the data center varies with CPU utilization (Wang et al., 2022b), the power consumption of the host $P\left(U_i\right)$ is derived from Eq. (1). (1) $$P\left(U_i\right)=P_i^{idle}+\left(P_i^{max}-P_i^{idle}\right)\times U_i$$where $P_i^{max}$, $P_i^{idle}$, $U_i$ present maximum power of host when experiencing full CPU utilization, minimum power of host with sleep state and host’s CPU utilization respectively. Energy consumption of host $(E_i)$ came from Eq. (2), thus the total energy consumption of all active hosts in the cloud data center $(E_{total})$ stems from Eq. (3). (2) $$E_i={\int }_{T_1}^{T_2}P\left(U_i(T)\right)dT$$ (3) $$E_{total}=\sum _{i=1}^M{E}_i$$

When the VM migration module is triggered. The average performance degradation of the VMs affected by migration is roughly 10% of the CPU utilization of the VMs (Wang et al., 2022b). Therefore, the overhead of VM migration $P{F}_j^{degradation}$ is defined as follows: (4) $$P{F}_j^{degradation}=\frac{1}{10}\times {\int }_{t_0}^{t_0+t_j^{mig}}{U}_j(t)dt$$ (5) $$t_j^{mig}=\frac{v_j^{Ram}}{h_i^{BW}}$$where the $t_j^{mig}$ is calculated by Eq. (5). Where, $U_j$ (t) presents the CPU utilization of VM $v_i$ at time $t$, $v_j^{Ram}$ expresses the RAM resource request capacity of $v_i$ and $h_i^{BW}$ represents the available BW resource capacity of host $h_i$.

The formulation of balanced resource allocation

Unbalanced resource allocation of hosts means that hosts with high resource utilization in single or multiple resource dimensions, and the available resources are not sufficient to allocate to the migrated VMs to ensure their working, which eventually leads to resource loss of the hosts. Thus, a balanced allocation of host resources can further improve the efficiency of resource allocation in cloud data centers and guarantee their quality of service.

CPU utilization accounts for a large proportion of hosts’ energy consumption, and RAM and BW are closely bound up with service level agreement violation (SLAV). Hence, the physical resources (CPU, RAM, BW) utilization reflects (Wang et al., 2022b) the impact on working performance of VMs to some extent. To evaluate the available resource parallelism that targeted host allocates resources for migrated VM with the account of CPU, RAM, BW (Ferdaus et al., 2014). In 3-D resource as shown Fig.1, the allocated resources flat surface of host $h_i$ according to real-time resources utilization, $\mathrm{\Delta }hos{t}_i^{allocated}$, is determined and denoted by Eq. (6), while the total resources flat surface of host $h_i$, $\mathrm{\Delta }hos{t}_i^{total}$, is presented by Eq. (7) respectively.

(6) $$\mathrm{\Delta }hos{t}_i^{allocated}:1=\frac{CP{U}_i^{allocated}}{CPU}+\frac{RA{M}_i^{allocated}}{RAM}+\frac{B{W}_i^{allocated}}{BW}$$ (7) $$\mathrm{\Delta }hos{t}_i^{total}:1=\frac{CP{U}_i^{total}}{CPU}+\frac{RA{M}_i^{total}}{RAM}+\frac{B{W}_i^{total}}{BW}$$

If the angle between the host’s allocated resource flat surface and the host’s overall resource flat surface is smaller and closer to parallel, it means that the host’s resources are more equally allocated. Thus, the resources allocation balance ratio denoted by $Balanc{e}_i^{ser}$ is defined as the cosine value between flat surface $\mathrm{\Delta }hos{t}_i^{total}$ and flat surface $\mathrm{\Delta }hos{t}_i^{allocated}$. The balance of resource allocation for $hos{t}_i$ is inversely proportional to the value of $Balanc{e}_i^{ser}$, with a smaller value of $Balanc{e}_i^{ser}$ indicating a more balanced allocation of the host.

Now we give the solution to the computation of normal vector of flat surface $\mathrm{\Delta }hos{t}_i^{total}$ and $\mathrm{\Delta }hos{t}_i^{allocated}$ defined as Eqs. (8) and (9). We assume that $\overrightarrow{Norma{l}_i^{total}}$ and $\overrightarrow{Norma{l}_i^{allocated}}$ denoted by normal vector of flat surface $\mathrm{\Delta }hos{t}_i^{total}$ and $\mathrm{\Delta }hos{t}_i^{allocated}$ respectively, and the value $Balanc{e}_i^{ser}$ is calculated below: (8) $$\overrightarrow{{\mathrm{N}\mathrm{o}\mathrm{r}\mathrm{m}\mathrm{a}\mathrm{l}}_i^{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}}}=\left(B{W}_i^{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}}-CP{U}_i^{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}}\right)\ast \left(RA{M}_i^{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}}-CP{U}_i^{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}}\right)$$ (9) $$\overrightarrow{{\mathrm{N}\mathrm{o}\mathrm{r}\mathrm{m}\mathrm{a}\mathrm{l}}_i^{\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{o}\mathrm{c}\mathrm{a}\mathrm{t}\mathrm{e}\mathrm{d}}}=\left(B{W}_i^{\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{o}\mathrm{c}\mathrm{a}\mathrm{t}\mathrm{e}\mathrm{d}}-CP{U}_i^{\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{o}\mathrm{c}\mathrm{a}\mathrm{t}\mathrm{e}\mathrm{d}}\right)\ast \left(RA{M}_i^{\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{o}\mathrm{c}\mathrm{a}\mathrm{t}\mathrm{e}\mathrm{d}}-CP{U}_i^{\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{o}\mathrm{c}\mathrm{a}\mathrm{t}\mathrm{e}\mathrm{d}}\right)$$ (10) $$Balanc{e}_i^{ser}=cos<\overrightarrow{Norma{l}_i^{total}},\overrightarrow{Norma{l}_i^{allocated}}>=\frac{(\overrightarrow{Norma{l}_i^{total}}\times \overrightarrow{Norma{l}_i^{allocated}})}{(\mid \overrightarrow{Norma{l}_i^{total}}\times \overrightarrow{Norma{l}_i^{allocated}}\mid )}$$

The proposed approach

We assume that the data center contains a set of heterogeneous hosts $H=\left\{h_1,h_2,\dots ,h_M\right\}$ ( $i\in <1,\dots ,M>$) and a set of VMs $V=\left\{v_1,v_2,\dots ,v_N\right\}$ ( $j\in <1,\dots ,N>$) and each host also hosts multiple VMs, intuitively as shown in Fig. 2. In this article, we mainly take into consideration resource type CPU, RAM, BW. When a user submits a resource request to the cloud provider, the cloud data center will provide a real-time service to create the VM instance, which will consume the resources of the physical machine in terms of CPU, RAM and BW. A host exhibiting high resource utilization can impact the performance of the VM. This is because the running VMs co-compete for host resources to fulfill their variable workload demands. When the VMM manager module is triggered and communicates with the VMP manager, the VMP manager establishes a more appropriate mapping relationship between VMs and hosts. This is achieved by employing an AHP-based resource balance-aware VMP strategy. The overarching goals encompass minimizing energy consumption, reducing the number of additional VM migrations, and alleviating service level agreement violations within a cloud data center.

AHP-based virtual machine placement strategy

The analytic hierarchy process (AHP), one of the multi-attribute decision-making models (Saaty, 1990), is to decompose a complicated problem into a number of levels and a number of influential factors, then to hierarchize the influential factors and transfer factors in data-form. It uses mathematical methods to calculate the relative weights of a number of influences affecting the decision. Ultimately find the best solution to the problem. The overall process is to first identify the criteria that influence the decision and construct a hierarchical decision tree. Subsequently a judgement matrix is developed based on the decision objectives. Then calculate the relative weights of each criterion, and obtain the weight matrix of each criterion after passing the consistency test. A analytic hierarchy process (AHP) is used to solve the VM placement problem. Decision criteria affecting VM placement and their judgement matrices are first identified. Then the relative weights of these decision criteria are calculated and the optimal host is found for the migrated VMs based on the weights. The detailed steps are as follows.

Step 1: determining the decision criteria and the hierarchical decision tree

Firstly, a hierarchical model is constructed. The first targeted layer is to dynamic virtual machine placement with energy savings and QoS guarantees, it alleviates energy consumption and SLAV for cloud data centers in the execution of VM scheduling. The second layer denotes the decision layer with three main criteria: power consumption, resource allocation balance ratio and available resources, and the third layer represents the available physical hosts ( $h_i$). The decision tree composed of these three layers is shown in Fig. 3. When VMM communicates with the VMP manager, the VMP manager module executes scheduling with the following three criteria:

• the increased power consumption ( $Norpowe{r}_i$) of host $h_i$.

• resources allocation balance ratio ( $Balanc{e}_i^{ser}$) of host $h_i$.

• available resource ( $A{R}_i$) of host $h_i$.

In the virtual machine placement process, the AHP-based decision criteria include power consumption, available resources, and resource allocation balance ratio. This framework governs the dynamic execution of virtual machine placement (VMP) with the objectives of minimizing energy consumption, reducing the number of VM migrations, and mitigating the impact of additional VM migrations on SLA violations that may result in performance degradation.

We use $X_{ij}$ to represent the mapping relationship between the VM and the host, which is defined as Eq. (11). When $X_{ij}$ is 1 it means that VM $v_j$ is placed on host $h_i$. (11) $$X_{ij}=\{\begin{array}{c}1\prime if{v}_{j}placeon{h}_i\\ 0\prime ifotherwise\hfill \end{array}$$

Upon placing the migrated VM onto the designated host, there is a subsequent rise in both CPU utilization and power consumption. The elevated CPU utilization, along with the resulting fluctuations in power consumption, are represented as $u_i^{incre}$ and $powerDif{f}_i$, respectively. $powerDif{f}_i$ is computed using the Eq. (12). To further unify the calculation normalize $powerDif{f}_i$ to $Norpowe{r}_i$ using the Eq. (13). $Norpowe{r}_i$ as one of the three main decision criteria. (12) $$powerDif{f}_i=P(U_i+U_i^{incre})-P(U_i)$$ (13) $$Norpowe{r}_i=(1-\frac{1}{1+e^{-powerDif{f}_i}})$$

A host with more available resources provides performance guarantees for virtual machines and also reduces the number of virtual machine migrations. In comparison to the energy consumption generated by RAM and BW resources, CPU resources account for the largest proportion of energy consumption in cloud data centers (61%) (Kusic et al., 2009). Therefore the amount of CPU available resources ( $A{R}_i$) is used as one of the three main decision criteria in the VM placement process. $A{R}_i$ is calculated as Eq. (14). $h_i^{mips}$ and $v_j^{mips}$ represent the total CPU resources of host $h_i$ and the CPU resource requirements of the $v_j$, respectively. (14) $$A{R}_i=1-\frac{{\sum }_{j=1}^N{X}_{ij}\times u_j\times v_j^{mips}}{h_i^{mips}}$$

AESVMP algorithm

The monitoring procedures of the data center periodically monitor the working status of the servers. A host with high resource utilization will suffer from resource contention among the VMs working on it, resulting in performance degradation. Therefore, the management system triggers VM migration according to the AESVMP mechanism proposed in this article, as shown in Algorithm 2, to build a new mapping relationship between migrated VM and host in the cloud data center with the goals of energy saving, reduction of SLAV and the number of VMs migration.

The application of Algorithm 1 is to calculate the weigh matrix of the decision criteria using AHP method when VM explores the targeted host. First, the judgment matrix of decision criteria (A) and the number of dimensions of the matrix ( $n$) are used as the input of the algorithm. We define various variables (lines 2–5). The weights of each decision criterion are calculated based on the judgment matrix to obtain the weight matrix ( $\overline{W}$) (lines 7). The normalizing weight matrix (W) is obtained through Eq. (15) (lines 10). Then ${\lambda }_{max}$, C.I. and C.R. are then calculated and used for the consistency test (lines 12–17). Output the standard weight matrix (W) after the final consistency test is passed (lines 18–23).

Algorithm 2 (AESVMP) illustrates the process of VM placement based on the Algorithm 1. First, the inputs to the algorithm are a list of migrated VMs and a list of hosts. Exclude hosts that are overloaded and dormant in (lines 8–13). The next step continues only if the condition is met that the available resources of the host exceed the requested resources of the migrated VM (line 14). Then, find the targeted host with the maximum score according to the calculation of Eqs. (10)–(20) (lines 15–19) for the migrated VM. Finally, return the result of the mapping relationship between VMs and hosts.

Time complexity analysis: We assume that the number of N migrated VMs and a set of M hosts are selected, the time complexity of performing a descending sort is $O(M{\log }_{}M)$. When triggering VMP, it is clear that the time complexity of targeted host is $O(N)$, so the time complexity of Algorithm 2 is $O(M{\log }_{}M+NM)$, and in the worst case when M equals N, the time complexity is $O(N^2)$.

Experimental environment

The proposed approach is to validate its performance under CloudSim emulator (Calheiros et al., 2011) in this article. In the tool, we simulate 800 heterogeneous host with two types HP ProLiant ML110G4 (Intel Xeon 3040) and HP ProLiant ML110G5 (Intel Xeon 3075) equally. The two types with same characterize of the number of CPU cores, RAM, BW and storage but are different in CPU capacity with 1860 MIPS and 2660 MIPS, respectively. The relationship between energy consumption and CPU utilization of the host is shown in Table 6. Then, four types of Amazon EC2 VMs, specific information as shown in Table 7, and PlanetLab project with 10 workloads, specific information as shown in Table 8, are taken into consideration during the experiment.

Evaluation metrics

For the experimental results, the following mainstream performance indicators (energy consumption, the number of virtual machine migration, SLA time per active host (SLTAH), perf degradation due to migration (PDM), service level agreement violation (SLAV) and aggregate indicators of energy consumption (ESV)) are determined to evaluate the performance of the proposed algorithm. These performance indicators are described below:

1. Where energy consumption represents the total energy consumption generated (Garg, Singh & Goraya, 2018) by all hosts running simulated workloads in the cloud data centers.

2. The number of VM migrations means the total number of VM migrations performed during the experiment. If the data center detects a host with a overload or underload state, then it starts VM migration. VM migration affects the performance of VM workloads. Therefore, the fewer VMs migrated, the better.

3. The working performance of migrated VM will be affected due to VM migration technology triggering, thus the performance degradation denoted by PDM is defined below: (21) $$PDM=\frac{1}{N}{\sum }_{i=1}^N\frac{P{F}_j^{\mathrm{d}\mathrm{e}\mathrm{g}\mathrm{r}\mathrm{a}\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}}}{C_j^{\mathrm{d}\mathrm{e}\mathrm{m}\mathrm{a}\mathrm{n}\mathrm{d}}}$$where N, $P{F}_j^{degradation}$ and $C_j^{demand}$ presents the number of VM, performance degradation and total CPU capacity of $v{m}_j$, respectively.

4. The user submits a request to create a VM instance to the cloud data center and signs a service level agreement with the cloud vendors. As defined in Beloglazov & Buyya (2012), service level agreements refer to the ability of the host and the previously recommended software measurement environment to meet the business quality requirements. SLA time per active host (SLATAH) indicates the percentage that the time of host with 100% CPU utilization divides the time of an active host. (22) $$SLATAH=\frac{1}{M}{\sum }_{i=1}^M\frac{T_i^{\mathrm{o}\mathrm{v}\mathrm{e}\mathrm{r}}}{T_i^{\mathrm{a}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}}}$$where M, $T_i^{over}$ and $T_i^{active}$ presents the number of active host, the time of host experiencing 100% CPU utilization and the time of active host, respectively.

5. SLAV is a indicator, service level agreement, to evaluate the overloaded host and performance degradation in combination with SLATAH and PDM. (23) $$SLAV=SLATAH\times PDM$$

6. ESV is an metric in association with total energy consumption ( $E_{total}$) and service level agreement violations (SLAV) and is calculated below: (24) $$ESV=E\times SLAV$$

Comparison benchmarks

To validate the efficacy of the method proposed in this article, we employed five distinct host state detection methods (THR, IQR, LR, MAD, LRR) and two VM migration techniques (minimum migration time—MMT, maximum correlation—MC) within CloudSim. These were utilized to conduct a comprehensive comparative analysis of the experimental outcomes involving the AESVMP algorithm, the PABFD algorithm (Beloglazov & Buyya, 2012), and the LBVMP algorithm (Wang et al., 2022b). For the sake of comparison, we computed the average results obtained from the five distinct host state detection methods (THR, IQR, LR, MAD, LRR). The safety parameter was set to 1.2 for IQR, LR, LRR, and MAD, while for THR, it was set to 0.8. All the comparative experiments were conducted using CloudSim, with a workload derived from 10 PlanetLab instances.

Experimental results

In this section, 10 workloads (Park & Pai, 2006; Chun et al., 2003) and the performance metrics mentioned are used evaluate the performance of the proposed AESVMP algorithm compared with the VM placement algorithm discussed above.

Table 9 shows the simulation results of the performance comparison between the AESVMP algorithm proposed in this article and the state-of-the-art LBVMP algorithm (Wang et al., 2022b) with the same condition. Where compared with the LBVMP algorithm the AESVMP algorithm outperforms in terms of the number of VM migrations, SLAV, ESV and Energy efficiency, with an average optimization of 51.76%, 67.4%, and 67.6% respectively, but AESVMP performs worse than LBVMP when it comes to energy consumption. We can conclude that the approach effectively optimizes in number of VM migration and the QoS.

Evaluation based on energy consumption

Figure 4 shows the total energy consumption generated by different methods in combination with two VM selection algorithms MMT and MC. When the VM selection methods are MMT and MC, the average energy consumption of AESVMP strategy is reduced by 27.9% and 27.7% compared to PABFD strategy, respectively. AESVMP takes into consideration criteria $Norpowe{r}_i$ and $Balanc{e}_i^{ser}$ to select the host with high energy-efficiency and underlines resource allocation balance, which can reduce energy consumption. However, the AESVMP strategy has a slightly higher energy consumption than the LBVMP strategy. This may be due to the fact that AESVMP, when selecting hosts with the same conditions, prioritizes hosts with more available resources and a more balanced resource allocation. It focuses on meeting the resource requirements of VMs to improve the quality of service. As a result, the data center employing the AESVMP strategy has a higher number of active hosts compared to the one using the LBVMP strategy, leading to a slight increase in energy consumption. While LBVMP is more focused on energy consumption optimization

Evaluation based on number of migrations

Figure 5 depicts a comparison of the performance metrics (the number of VM migrations) for the PABFD, LBVMP, and AESVMP strategies. When comparing the AESVMP strategy to the PABFD strategy, there is an average reduction of 68.5% and 73.1% when using the VM selection algorithms MMT and MC, respectively. Similarly, when comparing the AESVMP strategy to the LBVMP strategy, there is an average reduction of 57% and 52.7% when using the VM selection algorithms MMT and MC, respectively. The AESVMP strategy considers the available resource criteria of the host and ensures the fulfillment of resource requests from virtual machines. Consequently, this approach proves effective in reducing the number of additional VM migrations.

Evaluation based on PDM

Figure 6 depicts a comparison of performance metrics (PDM) for the PABFD, LBVMP, and AESVMP strategies. When comparing the AESVMP strategy to the PABFD strategy, there is an average reduction of 74.4% and 73.1% when using the VM selection algorithms MMT and MC, respectively. Similarly, when comparing the AESVMP strategy to the LBVMP strategy, there is an average reduction of 52.3% and 53.7% when using the VM selection algorithms MMT and MC, respectively. This reduction in the number of migrations directly contributes to the decline in PDM values. Consequently, it highlights the effectiveness of the AESVMP strategy in optimizing both the number of migrations and the PDM metric.

Evaluation based on SLATAH

Figure 7 demonstrates variations in the SLATAH performance metric using different methods. In an intuitive comparison, the AESVMP strategy, in contrast to the PABFD strategy, achieves an average reduction in SLATAH of 73.5% and 67.1% when the VM selection methods are MMT and MC, respectively. Similarly, when comparing the AESVMP strategy to the LBVMP strategy, there is an average reduction of 49.6% and 54.7% when using the VM selection algorithms MMT and MC, respectively. The AESVMP strategy prioritizes the criteria of available resources and resource allocation balance ratio. By selecting hosts with sufficient resource capacity and emphasizing balanced resource allocation, this approach reduces the likelihood of hosts becoming overloaded. Consequently, the SLATAH metric experiences a notable decline.

Evaluation based on SLAV

Figure 8 illustrates the SLAV performance metrics for various methods. When comparing the AESVMP strategy to the PABFD strategy, there is an average reduction of 94.2% and 90.3% when using the VM selection algorithms MMT and MC, respectively. Similarly, when comparing the AESVMP strategy to the LBVMP strategy, there is an average reduction of 74.1% and 77.4% when using the VM selection algorithms MMT and MC, respectively. This optimization of SLAV is directly linked to the reduction in both PDM and SLATAH. Thus the AESVMP strategy significantly reduces SLA violations.

Evaluation based on ESV

Figure 9 illustrates the performance metric ESV in relation to different methods. Intuitively, when compared to the PABFD strategy, the AESVMP strategy results in a reduction of ESV by an average of 95.7% and 92.8% when the VM selection methods are MMT and MC, respectively. Similarly, when compared to the LBVMP strategy, the AESVMP strategy reduces ESV by an average of 69.7% and 73.2% with VM selection methods such as MMT and MC. This substantial reduction in ESV is closely tied to energy consumption and SLAV.