Randomized routing of virtual machines in IaaS data centers
 Published
 Accepted
 Received
 Academic Editor
 Lan Wang
 Subject Areas
 Computer Networks and Communications, Optimization Theory and Computation
 Keywords
 Optimization, Cloud computing, Placement, Energy consumption, Service level agreement, Virtualization
 Copyright
 © 2019 Khaniand Khanmirza
 Licence
 This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. For attribution, the original author(s), title, publication source (PeerJ Computer Science) and either DOI or URL of the article must be cited.
 Cite this article
 2019. Randomized routing of virtual machines in IaaS data centers. PeerJ Computer Science 5:e211 https://doi.org/10.7717/peerjcs.211
Abstract
Cloud computing technology has been a game changer in recent years. Cloud computing providers promise costeffective and ondemand resource computing for their users. Cloud computing providers are running the workloads of users as virtual machines (VMs) in a largescale data center consisting a few thousands physical servers. Cloud data centers face highly dynamic workloads varying over time and many short tasks that demand quick resource management decisions. These data centers are large scale and the behavior of workload is unpredictable. The incoming VM must be assigned onto the proper physical machine (PM) in order to keep a balance between power consumption and quality of service. The scale and agility of cloud computing data centers are unprecedented so the previous approaches are fruitless. We suggest an analytical model for cloud computing data centers when the number of PMs in the data center is large. In particular, we focus on the assignment of VM onto PMs regardless of their current load. For exponential VM arrival with general distribution sojourn time, the mean power consumption is calculated. Then, we show the minimum power consumption under quality of service constraint will be achieved with randomize assignment of incoming VMs onto PMs. Extensive simulation supports the validity of our analytical model.
Introduction
InfrastructureasaService (IaaS) cloud providers (CPs), such as Amazon, Google and Microsoft, have huge data centers to provide on demand virtual machines (VMs) to their customers. An important issue for such data centers is to determine the server to which an incoming VM should be placed in order to optimize a given performance criterion. The CP has a variety of challenges, such as higher resource utilization, less cooling expenses and lower operation expenses. Fortunately, all of these efficiency metrics are positively correlated. Less power consumption means less operational expense, less cooling bills and higher utilization in the data center. This lets us choose the power consumption as the key metric representing others. On the other hand, cloud users who run their applications on VMs have their own concerns with quality of service. The resource management of CP has the chance to revise the initial placement of VMs onto PMs by live migrating techniques or dynamic consolidation. Considering live migration, the problem of VM placement can be divided in two parts as pictured in Fig. 1.

The routing of arriving VMs onto PMs

The optimization of the current VM placement by VM migration
According to this scenario, a VM request transmitted by a user to the data center is routed to the proper PM in the first step and then its placement can be optimized later.
The optimization of the current VM placement in the data center is analogous to the NPhard “Bin Packing Problem.” In this problem, a given set of items of variable size should assigned to the minimum number of bins taken from a given set. The VMs experience dynamic workloads, which means that the resource usage by a VM arbitrarily varies over time. In fact, the data center resource manager does not have the complete knowledge of the future resource usage (size) of VMs. The placement of VMs is monitored continuously and is tuned through the migration procedure. The virtualization technology let VM migrates (moves) between PMs on the fly. The migration of VM can be advantageous either when the resources utilization is too low, meaning that the PM is highly underutilized, or when it is too high, possibly causing overload situations and service level agreement violations (SLAVs). The optimization problem of VM placement problem is so complex that centralized and deterministic solutions are practically useless in large data centers with hundreds or thousands of servers as shown in several researches like (Wang & Gelenbe, 2015; Shojafar, Cordeschi & Baccarelli, 2016; Wang, Jiang & Wu, 2016). The centralized and deterministic algorithms may be appropriate in data centers with a limited number of servers, but may become inefficient in large and very large data centers, due to the complexity of the problem and the need for the simultaneous migrations of a large number of VMs.
These decentralized approaches have some side effect on routing procedure in the first part. The router does not have complete knowledge of the current placement of cloud data center. It is noteworthy that the router neither does not know about the size, the exact arriving time and sojourn time of the future VMs. These facts justify the stochastic modeling and analyzing of the router performance.
In this paper, we focus on the first problem: the problem of routing arriving VM to the host. We calculate probability of SLAV as well as total power consumption in a cloud data center using tools of queueing theory. A cloud data center differs from traditional queueing systems. First, a cloud data can have a large number of PMs; traditional queueing analysis rarely consider system of this size. Second, VM sojourn time must be modeled by general distribution instead of convenient exponential distribution. These differences pose significant challenges to the analysis. We use a novel approach to respond to these challenges. Our contributions in this paper are:

We model the cloud data centers as a group of M/G/n/n queuing systems with single task arrivals and a task buffer of finite capacity;

We define a novel optimization problem to minimize the power consumption under an explicit QoS goal for any VM consolidation system;

We find the optimal routing policy using numerical methods.
Analytical results are validated through discrete event simulation. Then, we compare our result with some benchmark algorithm for Google workload. The remainder of the paper is organized as follows. The “Related Work” section gives an overview of existing work on cloud performance evaluation and performance characterization of M/G/n/n + r queuing systems. It also introduces some heuristic algorithms for VM consolidation that we use for comparison. In the “System Model” section we discuss our analytical model in detail. We solve our optimization problem in order to obtain desired performance metrics in the “Optimization Problem” section. In the “Simulation Results” section, we present and discuss analytical as well as simulation results. We conclude the paper with the section “Conclusion” discussing the results and future research directions.
System Model
Consider a IaaS data center consisting of a set of P PMs. Let P = {1,2,...,P} denote index set of the set of PMs. Users request VMs to the provider. New request is either admitted or rejected in the admission control phase. An admitted request moves to the placement phase, where it will be assigned to one of PMs (Carvalho, Menasce & Brasileiro, 2015). We suggest a randomized router after the admission. The router sends incoming VMs to PM i with probability r_{i}, for all i ∈ P. The vector $\overrightarrow{r}=\{{r}_{1},{r}_{2},\dots ,{r}_{\leftP\right}\}$ is a probability vector and satisfies ${\sum}_{i\in P}}{r}_{i}=1$. VMs have independent and identically distributed service (sojourn) time with mean 1/μ. In addition, these sojourn times are independent of the host load. Assume that the utilization demand of all VMs is equal to one. Extensive analysis of huge data centers shows that majority of the VMs have approximately the same utilization (Reiss, Wilkes & Hellerstein, 2011). This observation supports our assumption. Assume that each PM only hosts at most n VMs. In the case a VM is assigned to an already full PM, the PM is overloaded. An overloaded PM degrades the QoS of the end user and we can assume this event as an SLAV (Domanal & Reddy, 2013; Beloglazov & Buyya, 2012). It should be remembered that the router is after admission control and admitted VMs could not be queued. All VMs arrive at the data center according to Poisson process with rate λ, thereby VMs arrive at PM i according to Poisson process with rate λ_{i} = λ r_{i}, for all i ∈ P (Wang, Chang & Liu, 2015). These processes are independent (see Section 6.4 in Trivedi (2001)). The whole data center can be modeled as a group of independent M/G/n/n (also known as generalized Erlang loss system) systems that work in parallel.
Generalized Erlang loss system
It is well known that for an M/G/n/n system, the steadystate distribution exists and is given by the “product form” as below (Kelly, 1991; Kimura, 1983).
(1) $${\pi}_{k}=\{\begin{array}{cc}{\pi}_{0}{\left(\frac{\lambda}{\mu}\right)}^{k}\frac{1}{k!},& \text{for}k\le n\\ 0,& \text{for}kn\end{array}$$where π_{k} denotes the steadystate probability that there are k VMs in an M/G/n/n system, that is ${\pi}_{k}={\mathrm{lim}}_{t\to \infty}{\pi}_{k}(t)$, and the steadystate probability that M/G/n/n is empty (π_{0}) is given by (2) $${\pi}_{0}={\left[{\displaystyle \sum _{k=0}^{n}{\left(\frac{\lambda}{\mu}\right)}^{k}}\frac{1}{k!}\right]}^{1}$$
Specifically, π_{n} describes the fraction of time that the PM is fully utilized. We call this probability SLAV and it is given by (3) $${\pi}_{n}=\frac{\frac{{(\lambda /\mu )}^{n}}{n!}}{{\displaystyle {\sum}_{k=0}^{n}\frac{{(\lambda /\mu )}^{k}}{k!}}}$$
Power consumption
We are interested in minimizing the total power consumption of the data center. According to the results of the standard experiments stated in (spec.org, 2018) (Fig. 2), the instantaneous power consumption of PM i is a function of utilization level of that PM (k) as below $${W}_{i}=\{\begin{array}{cc}a+bk,& \text{for}k0\text{\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}}(4)\\ 0,& \text{for}k=0\text{\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}}(5)\end{array}$$where 0 ≤ k ≤ 100 is integer variable and both a and b are fixed values. In our model, k can be seen as the number of VMs in PM i. Then the expected steady state of power consumption will be (6) $$\text{Exp}[{W}_{i}]={\displaystyle \sum _{k=1}^{100}}{W}_{i\mathrm{,}k}{\text{\pi}}_{i\mathrm{,}k}$$ (7) $$=a(1{\pi}_{i,0})+b{\displaystyle \sum _{k=1}^{100}k{\pi}_{i,k}}$$where π_{i,k} denotes the steadystate probability that the utilization of PM i is k and can be calculated by Eq. (1) for n = 100. Note that Exp[W_{i}] = 0 if and only if λ_{i} = 0 (π_{i,0} = 1). Our objective is to determine the vector $\overrightarrow{r}$ that minimizes the total expected steadystate power consumption of the data center, that is, (8) $$\underset{\begin{array}{l}\to \\ r\end{array}}{\mathrm{min}}{\displaystyle \sum _{i\in P}\text{Exp}[{W}_{i}]}$$
SLA constraint
We are interested in keeping the probability of SLAV below a given value ε. SLAV happens when a PM i does not have sufficient capacity for a new arriving VM (when M/G/100/100 is in state 100). The SLAV constraint is (9) $$\mathrm{Pr}(\text{SLAV})={\text{\pi}}_{i\mathrm{,100}}\le \text{\epsilon \hspace{1em}\hspace{1em}}\forall i\in P$$where π_{i,100} denotes the steadystate blocking probability for PM i.
Optimization Problem
In this section, we consider the optimization problem, in which the router decides where each incoming VM will be sent, so as to minimize the total expected power consumption subject to the SLA constraints. The optimization problem can be formulated as follows: (10) $$\underset{\begin{array}{l}\to \\ r\end{array}}{\mathrm{min}}\text{\hspace{1em}}{\displaystyle \sum _{i\in P}\text{Exp}[{W}_{i}]}$$ (11) $$s.t.\text{\hspace{1em}}{\displaystyle \sum _{i\in P}}{r}_{i}=1$$ (12) $${\text{\pi}}_{i\mathrm{,100}}\le \text{\epsilon \hspace{0.17em}\hspace{0.17em}}\forall i\in P$$ (13) $${\text{\lambda}}_{i}=\text{\lambda}{r}_{i}\text{\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}}\forall i\in P$$
Let us rewrite the optimization problem by changing our optimization variable from $\overrightarrow{r}=\left\{{r}_{1},{r}_{2},\mathrm{...},{r}_{\leftP\right}\right\}$ to $\overrightarrow{x}=\left\{{x}_{1},{x}_{2},\mathrm{...},{x}_{\leftP\right}\right\}$ which is defined below (14) $${x}_{i}=\frac{{\text{\lambda}}_{i}}{\text{\mu}}=\frac{\text{\lambda}{r}_{i}}{\text{\mu}}\text{\hspace{1em}\hspace{1em}}$$
Using Eq. (14) and putting Eq. (1) for k = 100 in Eq. (12) we get (15) $$\frac{{x}_{i}^{100}}{100!{\displaystyle {\sum}_{k=0}^{100}\frac{{x}_{i}^{k}}{k!}}}\text{\hspace{0.17em}}\le \epsilon $$
If we can solve this inequality constraint for x_{i} then we have a simple inequality constraint in the form of x_{i} < f(ε). Fortunately, numerical methods can be used to solve it. We show numerical results β for some practical values of ε in Table 1. For example, in the first row, we have β = 100 for ε = 0.1. It means that if we send VMs with a rate below 100 (x_{i} < β = 100) to PM i then we will guarantee that the probability of SLAV is below 0.1 (Pr(SLAV) < ε = 0.1) for that PM. The equivalent optimization problem will be (16) $$\underset{\begin{array}{l}\to \\ r\end{array}}{\mathrm{min}}\text{\hspace{1em}}{f}_{o}(\overrightarrow{x})={\displaystyle \sum _{i\in P}\gamma ({x}_{i})}$$ (17) $$\text{s}.\text{t}.\text{\hspace{1em}}{\displaystyle \sum _{i\in P}{x}_{i}}=\frac{\lambda}{\mu}$$ (18) $${x}_{i}\le \text{\beta \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}}\forall i\in P$$in which (19) $$\text{\gamma}(x)=a+\frac{1}{{\displaystyle \sum _{k=0}^{100}}\frac{{x}^{k}}{k!}}\left(a+b{\displaystyle \sum _{k=1}^{100}}k\frac{{x}^{k}}{k!}\right)$$
SLAV probability (ε)  Maximum rate (β) 

0.1  100 
0.05  95 
10^{−3}  80 
10^{−4}  73 
10^{−5}  69 
The γ(x) can be obtained using Eqs. (1), (2) and (7) with ease. We set a = 22 and b = 0.73 for a PM equipped with Intel Xeon E3 processor (spec.org, 2018). Then, we can show that the first order derivative of γ(x) is positive (γ′(x) > 0) and the second order derivative of γ(x) is negative (γ″(x) < 0) for (0 ≤ x ≤ 100) with numerical methods.
Theorem 1. For any x and y (0 < y ≤ x ≤ β), there exists δ (0 < δ ≤ y and 0 < δ ≤ β − x), so that (20) $$\text{\gamma}(x+\text{\delta})+\text{\gamma}(y\text{\delta})\le \text{\gamma}(x)+\text{\gamma}(y)$$
Proof: γ″(x) < 0 means that the derivative (γ′(x)) is nonincreasing. The condition y ≤ x implies that the γ′(x) ≤ γ′(y). Using definition of derivative and 0 < γ′(x), we obtain (21) $$0<\frac{\text{\gamma}(x+\text{\delta})\text{\gamma}(x)}{\text{\delta}}\le \frac{\text{\gamma}(y)\text{\gamma}(y\text{\delta})}{\text{\delta}}$$
Multiplying the above inequality by δ and adding γ(y − δ) + γ(x) yields Eq. (20).
Let X denotes the set of elements of $\overrightarrow{x}$. We define $\Phi (\overrightarrow{x})=\{x\in X0<x<\beta \}$.
Theorem 2. The size of the subset Φ for minimal vector is at most one.
Proof: Consider $\overrightarrow{x}$ which satisfies Eqs. (17)–(18) and $\Phi (\overrightarrow{x})>$. In our proof, we define a method to transform $\overrightarrow{x}$ to $\overrightarrow{x\prime}$. Then we show the following properties for the transformation.

The value of f_{o} for the transformed $\overrightarrow{x}$ is not greater than the original $\overrightarrow{x}$ (22) $${f}_{o}(\overrightarrow{{x}^{\prime}})\le {f}_{o}(\overrightarrow{x})$$

The transformation has convergence property. If we repeat the transformation we reach $\overrightarrow{{x}^{*}}$ where $\Phi (\overrightarrow{{x}^{*}})\le 1$ and we could not apply transformation anymore.
First the definition of transformation, because $\left\Phi (\overrightarrow{x})\right>1$, we can find x_{i} and x_{j} from $\Phi (\overrightarrow{x})$ where 0 < x_{i} < β and 0 < x_{j} < β. Without loss of generality, assume 0 < x_{i} ≤ x_{j} < β. We have two cases for x_{i} + x_{j}: (1) x_{i} + x_{j} ≥ β, (2) x_{i} + x_{j} < β. In the first case, we define δ = β − y_{i} and then change only the value x_{i} and x_{j} in $\overrightarrow{x}$ to get $\overrightarrow{x\prime}$ as follows: (23) $${{x}^{\prime}}_{i}={x}_{i}+\text{\delta}=\text{\beta}$$ (24) $${{x}^{\prime}}_{j}={x}_{j}\text{\delta}$$
In the second case, we define δ = y_{j} and then change only the value x_{i} and x_{j} in $\overrightarrow{x}$ as follows (25) $${{x}^{\prime}}_{i}={x}_{i}+{x}_{j}$$ (26) $${{x}^{\prime}}_{j}=0$$
Note that ${{x}^{\prime}}_{i}+{{x}^{\prime}}_{j}={x}_{i}+{x}_{j}$ and the constraint Eq. (17) is still satisfied by $\overrightarrow{x\prime}$.
After defining transformation, we prove the first property. Using Theorem 1 with defined δ, we can conclude (27) $$\text{\gamma}({x}_{i}+\text{\delta})+\text{\gamma}({x}_{j}+\text{\delta})=\text{\gamma}({{x}^{\prime}}_{i})+\text{\gamma}({{x}^{\prime}}_{j})\le \text{\gamma}({x}_{i})+\text{\gamma}({x}_{j})$$
Adding unchanged elements of $\overrightarrow{x}$ to both sides of above inequality, yields Eq. (22).
For the proof of the second property, consider Eqs. (23) and (26). These imply that at least one of x_{i} or x_{j} will not be in the subset $\Phi (\overrightarrow{x\prime})$. Then the size of subset Φ is decreased by the transformation as follows (28) $$\text{\Phi}(\overrightarrow{x})\Phi (\overrightarrow{{x}^{\prime}})=\{\begin{array}{ll}1& \text{for}{x}_{i}\ne {x}_{j}\\ 2& \text{for}{x}_{i}={x}_{j}=\frac{\beta}{2}\end{array}$$
Because $\left\Phi (\overrightarrow{x})\right>0$ it will reach 1 or 0 eventually and more transformation is not possible. At this point, due to the first property Eq. (22) we reach a minimal vector.
Without loss of generality we assume that the elements of the minimal vector are ordered ${x}_{1}^{*}\ge {x}_{2}^{*}\ge \mathrm{...}\ge {x}_{\leftP\right}^{*}$. The minimal (solution) of Eqs. (17) and (18) will be (29) $${x}_{i}^{*}=\{\begin{array}{ll}\beta \hfill & \text{for}1\le i\le n*\hfill \\ \frac{\lambda}{\mu}\beta n*\hfill & \text{for}i=n*+1\hfill \\ 0\hfill & \text{for others}\hfill \end{array}$$where ${n}^{*}=\lfloor \frac{1}{\text{\beta \mu}}\rfloor $ is the number of PMs which must be filled completely (up to β). The remaining load (if exists) must be dispatched to the next PM. For large scale data centers, we can neglect this PM and show the solution of Eqs. (10)–(13) as follows.
(30) $${r}_{i}^{*}=\{\begin{array}{ll}\frac{1}{n*}& \text{for}1\le i\le n*\\ 0& \text{for others}\end{array}$$
For implementation, we only need a random generator. When a new VM arrives, we draw i form [1, n*] and sends this VM to PM i. We do not require any polling, therefore our implementation is simple and agile as we need it in a cloud data center.
Simulation Results
To validate the analytical solution, we have built a discrete event simulator of a CP data center using MATLAB (2012). We have considered the system with two different sojourn time distribution: exponential and uniform. In both cases the mean sojourn time is fixed at μ = 10^{1}. The mean inter arrival time of VMs was made variable from (λ = 10^{4} to 10^{8}) to give reasonable insight into the behavior and dimensioning of CP data centers. Regarding λ and μ, traffic intensity varies from 10^{4} to 10^{8} which represents the mean number of VMs in data center at steady state according to the Little formula. The number of active PMs according to $\left({n}^{*}\text{\hspace{0.17em}}=\text{\hspace{0.17em}}\frac{\text{\lambda}}{\text{\beta \mu}}\right)$ depends indirectly on ε, e.g., for ε = 10^{−5} (β = 69) it varies from 145 to 145,000 servers. The values chosen may be quite applicable to small to largesized CPs data centers that try to keep the utilization of their servers as high as possible while guarantee a minimum QoS for the users. It is noteworthy that no CP published information regarding average traffic intensity, number of servers or the percentage of reserved.
We generate confidence intervals (95%) for steady state measurement using independent replications with deletions method. First, we run 50 independent replications of each simulation, then we remove samples of transient phase and finally we calculate the sample mean. Figure 3 depicts the SLAV probability in data center at steady state. Simulation results follows analytical model perfectly for all (λ/μ) and β, values. This observation supports the validity of the analytical model findings. As can be seen, probability for exponential sojourn time is generally less than probability for uniform one. Note that the mean time is the same for both distribution, and this may relates to the variance. The variance of exponential is about 10, but the variance of uniform is about 30. As the number of active servers increases, the SLAV probability decreased steadily. This trend is due to the fact that with more active servers, an arriving VM has more places to be hosted and the chance of blocking and SLAV is lower.
Figure 4 shows the effect of sojourn time distribution on the convergence time. The data center with exponential sojourn time reaches to steady state sooner than a data center with uniform sojourn time. We only show the results for β = 100, λ = 10^{5}, μ = 10 because the discussion aims to highlight the influences of distribution on convergence time; for larger systems and lower probability, the discussion is the same.
In order to study the power consumption of our method in practice, we use Google trace (Reiss, Wilkes & Hellerstein, 2011) which consists of several concurrent VMs in a single ∼12K server farm where the resource demand of VMs is highly dynamic and needs quick decisions. Figure 5 compares the power consumption for our method with two benchmark algorithms in the literature: PABFD (Beloglazov & Buyya, 2012) and MT (Domanal & Reddy, 2013). The power consumption in our method is just about 2% more than PABFD and MT. This higher power consumption is acceptable, because the workload is highly dynamic, varying over time, and is driven by many short jobs that demand quick scheduling decisions and updates. PABFD and MT suffers from long decision process and overwhelming migrations. These shortcomings make both of them fruitless in real world cloud computing scenarios. The comparison gives us an idea about how far we are from idealistic benchmark algorithms in the literature.
Conclusion
Effective resource management is a major challenge for the leading CPs (e.g., Google, Microsoft, Amazon). Performance evaluation of data centers is an important aspect of resource management which is of crucial interest for both CPs and cloud users. In this paper, we proposed an analytical model based on the M/G/n/n system for performance evaluation of a cloud computing data center. Due to the nature of the cloud computing, we assumed general sojourn time for VMs as well as large number of PMs. These assumptions make our model acceptable in terms of scale and diversity. Through extensive numerical simulations, we showed that our analytical model closely alignes with simulation results. Our results also indicate that our method consumes a bit more power than idealistic benchmark in the literature.
In the future, we plan to extend our model for variable size VMs. Studying how “Power of Two Choice” can improve the result of randomized dispatcher will be another dimension of extension.