2012 IEEE/ACM Fifth International Conference on Utility and Cloud Computing
A Representation Model for Virtual Machine
Allocation
Giacomo Mc Evoy, Fabio Porto and Bruno Schulze
National Laboratory for Scientific Computing (LNCC)
Petrópolis - Brazil
Abstract—In Cloud Computing, the utilization of virtual machines (VMs) as a computational resource unit is a fundamental
concept. In a Cloud environment, VMs are deployed on physical
resources with a set computational characteristics, such as processing power. VMs have computational properties as well, which
are drawn from their respective hosts, and may be migrated from
one host to another. In this context, the perceived performance
of a parallel application is determined by how the respective
VMs are allocated to the physical resources. For instance, two
VMs deployed in the same host may have their performance
altered in complex ways, due to resource contention. Also, the
efficiency of a application that spans many VMs is dependent on
the distribution of the correspondant processes, due to available
communication bandwidth between VMs, but also due to the
possibility of leveraging local communication between processes
or threads in the shared memory of a single host. As an attempt
to describe the complex conditions that influence the performance
of applications running on the same Cloud environment, we
propose a more detailed representation of VM allocation, with
features that can be used in the future for building more accurate
performance models.
of [3], conscious VM placement can significantly improve
performance when consolidating servers, as CPU bottlenecks
are managed by prediction of resource consumption. However,
this approach is limited to workloads constrained to a single
VM, without inter-VM communication. As suggested in [4],
the actual physical consumption of a VM is dependent on its
location and interactions with other VMs.
Consequently, a better understanding of the relationship
between the environment produced by concurrent VM allocations —both competing and collaborating— and perceived
application performance can thus be a path towards improved
performance models that are useful for executing HPC applications in the Cloud.
In this context, the main contributions of this ongoing work
are the following:
•
•
I. I NTRODUCTION
Cloud Computing is an emergent operational model for
scalable and flexible service provision, that has recently attracted interest in the scientific community. As computational
requirements of eScience applications continually increase,
mainly due to the need to solve computational problems
at larger domain scales, high-performance computing (HPC)
becomes an essential tool to several aspects of the scientific
endeavor. Clouds enable the deployment of infrastructures
with high scalability, aimed at tackling massive computing
problems, by leveraging the possibility of rapid deployment
and removal of virtual machines (VMs), a property called
elasticity [1].
According to the technical report in [2], the scientific
applications that gained the greatest benefit from Cloud Computing are pleasingly parallel applications, like those deployed
in the Cloud-enabled framework Hadoop, mainly because
bag-of-tasks are natural candidates for speed-ups in cloud
computing. However, the report also states that Cloud environments introduce an important challenge in the context of
application performance, and that scientific applications have
special requirements that require tailored solutions. We argue
that an important issue behind the relatively poor performance
of parallel scientific applications in Cloud environments is the
lack of performance models that can relate the nature of VM
allocation to perceived performance. As shown in the work
978-0-7695-4862-3/12 $26.00 © 2012 IEEE
DOI 10.1109/UCC.2012.51
•
Discuss the importance of modeling VM placement for
scientific applications running in Cloud environments.
Propose a conceptual model for the representation of
VM allocations that deploy parallel applications, using
the Object-Relationship (OR) approach, aimed to register
and analyze the properties and effects of concurrent, colocated deployments.
Develop model constraints based on predicates to ensure
consistency with realistic VM mappings, so that the
model contains valid VM distributions that can be later
be considered as scheduling candidates.
This paper is organized as follows. Section II discusses how
the subject of VM placement relates to the performance of
scientific applications. In Section III, we elaborate the scope
of our proposal by formally formulating the representation
problem. In Section IV, we present our proposed model to
register VM allocations and application deployment. Finally,
we conclude the paper in Section V, with possible future work.
II. BACKGROUND
The authors in [5] describe Cloud Provisioning as the process of deployment and management of applications in Clouds,
consisting of three steps: a) Virtual Machine Provisioning,
which involves instantiation of one or VMs that match specific
hardware and software requirements of an application; b)
Resource Provisioning, which is the mapping and scheduling
of VMs onto physical Cloud servers and c) Application Provisioning, which is the deployment of applications within VMs
and mapping of end-user’s requests to application instances.
247
271
In pubic IaaS Clouds such as Amazon EC21 , Resource
Provisioning is entirely managed by the Cloud Provider (CP),
leaving some control of VM Provisioning to the client.
Scheduling in Amazon EC2 is probably oriented towards
server consolidation that minimizes resource consumption,
while still ensuring availability. This approach has negative
impacts in application performance, due to unmitigated virtualization overheads, as studied in works such as [6] and [7].
Both works discuss the performance degradation of computeintensive applications in a public Cloud, specifically on performance variability, which can affect the reproducibility of
scientific in-silico experiments.
While some performance penalties can be mitigated by
careful Application Provisioning —e.g. under-provisioning
of resources during the deployment of processes [7]— and
VM Provisioning —e.g. determining optimal VM count and
number of cores [8], [7]—, Resource Provisioning can still
play an important role in private Clouds, whenever the VM allocation process can be actually controlled. In this context, the
scheduling process is driven by performance models that take
into account VM capacity planning and VM placement [9]. As
presented in [10], resource consumption and performance of a
given process are not fixed, they depend of the prevailing VM
allocation. Performance of one application can be influenced
by other concurrent applications that share physical resources.
There are some works that study the problem of VM
placement to improve perceived application performance. Do
et al. [11] define one set of performance metrics to characterize
resource usage and another set to characterize application
performance, and use canonical correlation analysis (CCA) to
analyze the correlation between the two sets of variables and
achieve workload consolidation that can sustain performance.
Wood et al. [12] studies a black-box approach with resource
utilization for hotspot detection, so that VMs that generate
performance bottlenecks can be migrated after initial allocation. Bobroff et al. [13] introduce an empiric dynamic server
migration and consolidation algorithm based on predicting
capacity demand of virtual servers using time series analysis.
Garg et al. [3] classifies workloads and proposes allocation
heuristics based on the type of workload and the current
load that a server has, seeking consolidation while support
performance SLOs.
Consolidation strategies can then be classified as a) adaptive
VM migration through observation of resource consumption
and b) estimation of application performance via a performance model. However, the strategies that we found in the
literature are based on a key assumption: the execution context
is constrained to a single VM. By this we mean that each set
of processes running inside each VM are assumed to not be
communicating with other VMs, either because the workloads
are independent or are very weakly coupled, as in Bag of Tasks
(BoT) applications and load balanced Web servers. While this
may be a viable assumption for applications with infrequent
inter-VM communications, as is the case for high-throughput
1 Amazon
computing (HTC) applications, performance analysis in [7],
[8] and [14] show that this assumption cannot be sustained
for applications that are not embarrassingly parallel, thus
many HPC scientific applications show poor scalability. A
previous work [15] was an attempt to assess how performance
of scientific applications is suceptible to: a) many types of
resource contention; b) patterns of inter-VM communication
and c) interfering and collaborating VM co-location.
In this context, we argue that performance models aimed at
HPC Resource Provisioning should consider the distribution
of VMs —which hardware elements are being shared by VMs
and which VMs form virtual clusters— in the more general
scenario of co-locating several workloads of collaborating
VMs in a shared environment. This ongoing work addresses
the problem of providing a representation model for the
allocation of groups VMs, as an initial step towards scheduling
algorithms that consider VM placement.
III. P ROBLEM F ORMULATION
The allocation of VMs to physical resources is an essential aspect of Cloud Provisioning. Given a request to a
Cloud environment, where n VMs are to be instantiated,
the allocation process (scheduling) consists in assigning one
physical machine (PM), from the set of available PMs, to
each requested VM. In order to build a conceptual model that
describes the process of VM allocation with an OR approach,
the process needs first to be formally formulated; this implies
restricting the definition of what is considered a proper VM
allocation, as well as stating the extent of the model. We state
our assumptions for the representation problem as follows:
• Applications consist of a variable set of processes, which
are deployed in one or more VMs on execution. Each VM
hosts processes corresponding to a single application.
• Processes of a given application are either equal (SPMD
approach) or reflect the Master/Worker paradigm. A process may not migrated to other VMs once it executes, but
a VM may be migrated to a different host at runtime.
• Resource consumption of a VM is dependent on the
spatial distribution of other collaborating VMs.
• Process performance can be affected by resource consumption of other VMs in the same host.
• The mapping of a virtual core can be manually restricted
to a subset of the available physical cores.
The first assumption means that the model allows the general case of concurrent, multi-threaded applications deployed
in virtual clusters, with the restriction that VMs cannot be
shared by applications. Application processes end up executing
in virtual processor cores, which are mapped to physical
cores by the virtualization solution. The second assumption
simplifies the model, and can be relaxed in future refinements.
The third and fourth items state that the performance effects
of VM placement should not to be ignored by a performance
model, as discussed in the previous section.
At any instant during the execution of a VM, each virtual
core is allocated to exactly one physical core. Following
previous results of [15], The last item considers the capability
Elastic Compute Cloud, http://aws.amazon.com/ec2
272
248
of specifying virtual core mappings, or vcpu-pinning. Usually,
all physical cores in a server are eligible for a virtual core
allocation, but this can be changed —either via settings in the
virtualization solution or indirectly by process affinities where
applicable— to a subset of the available cores. With vcpupinning, the scheduler that manages virtual core allocation
chooses from one of the pre-specified candidates. This setting
may improve performance in some conditions, specially with
compute-intensive workloads [15]. Given a virtual core nv of
a VM, the assigned physical core nr throughout time t is given
by a mapping function, which can be stated as:
nr ∈ candidates (allocated)
S : map(nv , t) =
(1)
∅
(not allocated)
inside a virtualized environment. In particular, this work aims
to describe VM placement for private IaaS Clouds oriented
to HPC scientific applications. The purpose is to leverage
the representation model in the longer term goal of relating
overall VM distribution to perceived application performance.
We consider this a key step towards improving performance
of HPC applications in Cloud environments that focus on
throughput and response time instead of profit, such as expected of scientific Clouds that support either the private or
community service models, and that are designed on dedicated,
high-performance infrastructures.
In this work, we assume that the allocation of a VM is fully
defined by the allocation of each of its cores, as given by (1).
This representation may not be sufficient for other resources
besides processing power, such as the linking of storage capacity from different hosts to a VM. This representation is able to
capture the re-arrangement of virtual cores mappings inside the
same host, which can affect performance of compute-intensive
applications [15].
As discussed in the introduction, our goal is to gain an
understanding of the relationship between VM spatial distribution and perceived application performance. In order to
acquire such understanding, we propose acquiring and storing
details about the VM allocation of applications executed in
a multi-tenant environment. We expect that an application
execution database that can be queried a posteriori should
reveal evidence about the effect of VM placement, particularly
due to resource contention. This section discusses the OR
model of such database, including the entities, relationships
and constraints. It also presents a brief proof of concept of
populating the database to produce observable results.
The nature of scheduling requires an extension of the
traditional database model in order to support time durations
and temporal relationships. There is a solution from the
scientific community called MADS (Modeling of Application
Data with Spatio-temporal features) [16]. It is a conceptual
spatio-temporal model with multi-representation support, that
provides rich mechanisms —such as complex data types and
semantically enriched relationships— for describing the data
structure component of applications. The idea is to utilize
MADS in order to facilitate the following: a) produce a data
model for the spatio-temporal representation of VM allocation
and b) determine the model constraints, using logical predicates, that correspond to the restrictions that arise from the
nature of the core mapping.
IV. O UR P ROPOSAL
Fig. 1: Temporal diagram showing virtual core allocations.
Considering this formulation, this work proposes the development of a conceptual model able to describe: a) the
allocation of VM instances, b) the deployment of application
processes to these instances and c) some basic aspects of the
execution. These descriptions span through all three steps of
the Cloud Provisioning process, as presented in the previous
section. We label the spatio-temporal allocation of a group of
VMs related to the execution of a single application execution
as an execution placement. An execution placement encompasses the spatial distribution of all virtual cores relevant to
an application in a given deployment (which physical cores
are the virtual cores allocated to), as well as the temporal
allocation (how does these allocations change throughout
time). Figure 1 shows a diagram that captures the mapping
of all virtual cores of two VMs —v1 has one core and v2 has
two cores— throughout space in time. The diagram allows
the inference that v2 was migrated from the physical machine
m1 to m2 , and had its map settings changed afterwards. If
each VM is hosting one application, the diagram effectively
represents the two corresponding execution placements.
This work thus proposes a model to register several —
potentially concurrent— execution placements, taking place
A. Conceptual Model
We propose a conceptual model for the representation of
VM allocation, with the following considerations:
•
•
273
249
Focus on VM allocation. This work is not concerned with
representing the whole scheduling process of an application in a Cloud environment. The model is restricted
to represent the VM allocation of an application where
the VMs are already defined. Thus, an execution instance
already specifies the amount and profile of the VMs, as
well as the type and number of processes per VM.
VM instances are not reused in the model. In order to
facilitate the model’s consistency, each VM instance is
tied to a unique VM placement, i.e., different placements
imply different VMs. Also, each VM instance is tied to a
single execution in the model. As a consequence, a VM
•
•
•
processes. Each tuple in Exec Placement can be described
as the composition of all the tuples of VM Placement
that correspond to a given execution. Also, VM Placement
tuples can be related to each other to highlight concurrency.
This is established in the Concurrency relationship, which
uses temporal overlapping of two or more placements.
instance in the real world may be represented by many
entries in the model.
Use relevant core mappings. Based on the stated problem
formulation, the placement of a VM is expressed as a
composition of mappings of its virtual cores to physical
cores, throughout time. The model only registers effective
core mappings, meaning that the VM is in a running
state, and that the application processes are already executing. The details of the representation of a mapping is
explained later in this section.
Focus on CPU utilization. While this model can be used
to study various types of resource contention (CPU, memory, storage, network), we focus on how the physical CPU
cores are being utilized throughout time. As explained
in the previous item, the model provides information
about possible resource contention related to the CPU,
e.g. from different levels of caching. Interference from
I/O intensive applications may be inferred by correlating
it with current server consolidation, but the model does
not capture further details.
Coarse grain representation. The model does not currently describe actual scheduling of virtual cores between the eligible physical cores. Given a specified
mapping configuration, the virtualization solution may
choose different targets for a virtual core throughout time,
possibly many times within a second. In this solution,
representation of VM allocation is restricted to said mapping configuration. Likewise, the model ignores actual
scheduling within a VM by its OS, i.e. how the processes
are allocated to the virtual cores throughout time.
Fig. 3: Diagram with the machine-related entities and their
specializations.
Figure 3 shows the entities related to the physical and virtual
machines. The entity called Machine represents the common
concept of machine, either real or virtual. Two disjoint, allinclusive subtypes are provided, one for each type so that
each instance of machine is restricted to one or the two specializations. A similar strategy is applied to the Processor
and Core entities. Each base entity has a type attribute,
which can have the values {‘real’, ‘virtual’}, so that
specialized tuples have only one of these values in the attribute.
The part of the model related to the interactions between
physical and virtual machines is presented in Figure 4. The
aggregation relationship is used to express how a machine is
related to its processors and cores. The type attributed is used
to establish that a physical machine can only be composed of
physical components, as opposed to virtual ones. In this way,
physical and virtual hierarchies are separated.
The Core Placement entity is defined so that each tuple
represents a single mapping between a virtual core and a set
of physical cores during a continuous period of time. It is
defined as a temporal entity, highlighting the temporal nature
of the mapping. If a virtual core is mapped to a different
set of physical cores after some time, a new tuple must be
generated. The association cardinalities with the Virtual
Core indicate that a virtual core can participate in zero or
several allocations, where each allocation corresponds exactly
with one virtual core. The relationship with the Real Core
The model can be represented by a single relationship
graph between its entities, but it is split in three diagrams
for readability. Figure 2 shows a diagram that with the entities
related to application execution and VM placement. We define
the Execution entity as an instance of an application that
is run on a set of VMs for a period of time. Each tuple has
a temporal attribute that represents the time interval between
the beginning and end of the execution. It may also include
data about the performance obtained, and settings most related
to performance. Each application can be executed many times
throughout time —possibly concurrently—, as indicated by the
corresponding relationship. The Exec Placement entity
provides a description of the allocation of the VMs of a
specific execution throughout time, taking into account that
any VM may be suspended or migrated, effectively modeling
an execution placement. Each Exec Placement tuple is
related to zero or one Execution tuples, describing the
actual VM allocation for that execution. We stress that this
relationship is not meant to describe a scheduling selection
process, where an execution selects one of many allocation
candidates. The Exec Placement entity is described as
a temporal entity, with an IntervalSet time span. This
indicates that each tuple is associated with a set of time
intervals, which in the model corresponds to the intervals
where at least one VM of an execution is running one or more
274
250
Fig. 2:
Diagram showing the application execution and VM placement in the model.
straint deals with the fact that a virtual core can only be
attributed to one set of physical cores at a given time,
as established by manual settings in the virtualization
solution. If the virtual core is assigned to another set for a
later time interval, the two time intervals cannot overlap.
• Constraint 2: at any given time, all virtual cores of
a given VM should be allocated to the same physical
machine. This constraint is needed because a VM is
naturally instantiated in a single physical machine, but
the actual allocation is expressed via its cores. It thus
ensures a valid description of a VM allocation.
The following constraint is not strictly necessary for consistency, but it is pertinent to some Cloud environments, such
as Eucalyptus, which enforce this restriction:
• Constraint 3 (optional): in a given physical machine, the
total number of allocated virtual cores should not exceed
the number of real cores at any given time.
In order to express these constraints, several definitions
are needed. First, let the sets Mv , Pv and Nv be defined
as the collection of all possible tuples in the Virtual
Machine, Virtual Processor and Virtual Core entities,
respectively. Similar definitions for the physical entities follow
with Mr , Pr and Nr . We also define Gr as the set of
all possible groupings of real cores, such that each element
may participate in the GroupAllocates multi-association
relationship:
is a special type of association called multi-association. In general, it means that the association is done between groups of
tuples from each of the intervening entities, while also defining
how the groups are formed. As indicated by the cardinalities,
this association in particular relates each instance of Core
Placement with exactly one group of Real Core tuples.
A group of Real Core tuples may be associated with many
allocations, while each tuple of Real Core is associated to
zero or more groups. This enables a virtual core to be mapped
to any subset of the available physical cores.
With this representation, only actual core allocations are
recorded, i.e., if a virtual core is unallocated for a time interval,
then it is not registered for said interval. An exception is when
a tuple for a virtual core has just been added to the database,
a scenario that is being allowed to avoid the complications
of constraining a parent-child relationship where both participations are mandatory. A virtual core in the model without
allocation can then correspond to a VM that is not executing
any application process. Lastly, the Core Placement is
related to the VM Placement. The relationship is of the
aggregation kind, where a VM Placement tuple is to be
related to all the Core Placement tuples that correspond
to the virtual cores contained in a VM. By capturing core
allocations, the model infers from core allocations, VM and
execution placements using compositions.
With the proposed representation of core allocations,
uniqueness is assured for the mapping of a virtual core for
two exact time intervals. However, without model constraints,
it is possible to have an association between a virtual core and
two overlapping time intervals, which would render the model
inconsistent with reality. Model constraints are elaborated next.
Gr = {r = {r1 , . . . , rn }; r ∈ n ∈ N; ri ∈ Nr ∀i}
(2)
We use the notation Composes(Mv , Pv ) to denote the
set of tuples of the Composes aggregation relationship, applied between the Virtual Machine and Virtual Processor
entities, with an analogous notation for the other aggregations.
Then, it is possible to define the has() predicate for the
following cases:
B. Model Constraints
As described in the previous subsection, the conceptual
model’s representation does not guarantee the consistency of
the VM placement throughout time, thus the model needs to
be constrained to ensure consistency. This section discusses
two integrity constraints:
• Constraint 1: given a virtual core nv , no two of its
mappings should have temporal overlapping. This con-
mv .has(pv )
pv .has(nv )
⇐⇒
(mv , pv ) ∈ Composes(Mv , Pv )(3)
⇐⇒
(pv , nv ) ∈ Composes(Pv , Nv ) (4)
mv .has(nv )
⇐⇒
∃pv ; mv .has(pv ) ∧ pv .has(nv ) (5)
with an equivalent application for physical entities. Let It
be the timeline of observation, thus the map() predicate that
275
251
Fig. 4: Diagram showing the expression of VM allocation via core mappings.
in Core Allocation. In order to determine if the new
allocation is possible, we can use the maximum number of
virtual cores allocated in the same physical machine at any
time instant belonging to the interval in , called nmax . First,
determine the physical machine mr where nnv is allocated
(according to Constraint 2, mr is defined unambiguously).
Secondly, we define the set of physical cores of mr as
Nmr = {n ∈ Nr ; mr .has(n)}. Afterwards, determine the
allocations in mr whose time intervals overlap with in , and
derive the corresponding set of time intervals Iover :
Iover = i ∈ It ; ∃nv ∈ Nv ; overlaps(in , i);
map nv , {r1 , . . . , rl }, i ; rj ∈ Nmr ; ∀j = {1, . . . , l}
describes the temporal mapping between virtual and physical
cores is defined as:
map(v, r , i) ⇐⇒ (v, r, i) ∈ Core Allocation;
(6)
v ∈ Nv , r ∈ Gr , i ∈ It
The model allows virtual cores without allocations to facilitate minimum cardinality enforcement. We define the predicate
overlaps(i1 , i2 ) as being true if and only if the time
intervals i1 , i2 ∈ It overlap. This predicate can be extended
in the following way:
overlaps(i1 , . . . , in ) ⇐⇒ overlaps(i1 , i2 ) ∧ . . .
∧ overlaps(i1 , in ) ∧ . . .
∧ overlaps(i2 , i3 ) ∧ . . .
∧ overlaps(in−1 , in )
(7)
This allows the calculation of nmax :
i1 , . . . , inmax =
With those definitions presented, the constraints can be
elaborated as follows:
arg max
overlaps(i1 ,...,ik )
ij ∈Iover
k
(8)
Where the overlaps predicate is as defined in (7). In order
to insert the tn tuple, the constraint then requires
Constraint 1: Let v ∈ Nv . Then, i1 , i2 ∈ It with
map(v, r1 , i1 ) ∧ map(v, r2 , i2 ) for any r1 , r2 ∈ Gr ,
such that i1 .overlaps(i2 ). If v participates in one or zero
mappings, then the constraint is satisfied trivially.
n < |Nmr |
(9)
C. Proof of Concept
Here, we present a case study describing the deployment
process of an application in a virtualized environment using our proposed model. We assume that a database with
normalized tables has been constructed using the conceptual
model. The database is implemented through the conversion
of temporal features to natural SQL, which is done by MADS
automatically. The objective is to describe the process of
populating the model entities with values relevant from the
application provisioning, thus obtaining a representation of the
execution placement. Highlighting the actual influence of VM
placement by analyzing performance metrics is omitted due to
space constrains and left for future work.
Figure 5 shows a UML diagram describing in detail the
provisioning and execution processes for one scenario, along
with the proper interactions with the representation model, so
that the latter capture the details of the VM placement. We
Constraint 2: Let nv1 , nv2 ∈ Nv and mv ∈ Mv such
that mv .has(nv1 ) ∧ mv .has(nv2 ). Let r1 , r2 ∈ Gr ,
i1 , i2 ∈ It such that map(nv1 , r1 , i1 ) ∧ map(nv2 , r2 , i2 ) ∧
i1 .overlaps(i2 ). The constraint then requires the following:
1) ∃mr1 ∈ M such that mr1 .has(r1i ), with r1 =
{r11 , . . . , r1j }; i = {1, . . . , j}
2) ∃mr2 ∈ M such that mr2 .has(r2i ), with r2 =
{r21 , . . . , r2k }; i = {1, . . . , k}
3) These physical machines are the same, i.e. mr1 = mr2 .
If any of r1 or r2 are empty sets, then the constraint is
satisfied trivially.
Constraint 3: The constraint will be enforced when attempting to add the new tuple tn = nnv , {r1 , . . . , rk }n , in 276
252
assume that there is some integration between the execution
environment and the model database. The event-driven mechanisms behind the integration are not described due to space
constraints, leaving the details for future work. Nonetheless,
we provide detail of the three model interactions in Figure 5:
This first interaction (Create) is done after the VM placements for an execution have been defined in the environment,
and before processes begin execution. The application tuple
is created if it did not exist, and an execution tuple is added,
related to the former. New Virtual Machine tuples are
inserted related to the execution tuple, and the virtual hierarchy
in 3 is resolved. Tuples in Physical Machine are added
if not already present. Although the execution environment
already has details on core placements, mappings are only
registered while processes are executing.
During process execution, the user (or an external scheduler) may choose modify the current execution placement,
triggering a new interaction (Update). The event can be either
migrating the VM to another host or modifying core mapping
s in the virtualization solution. In any case, the time interval
of the previous set of core placements is now defined at
both ends, and can be added to Core Placement. This
is done by first defining or reusing a group of physical
cores, and relating it to one virtual core. Only for the first
interval, VM Placement and Execution Placement
tuples are created, while the compositions are updated for
every interval. The temporal property of VM Placement is
calculated to reflect all time intervals during which the VM
executed its processes. The temporal property of Execution
Placement is then calculated based on the former, to reflect
all time intervals during which at least one process of an
application execution was active. Concurrency of VMs are
added to Concurrency by inferring time overlaps so that
this can be queried later. If the VM was previously migrated,
the execution environment needs to provide the time where
application processes resumed execution.
When all application processes finish execution, an event
is triggered and a new model interaction (Finalize) happens. Here, the last allocation intervals are determined: completely analogous to the Update interaction, new tuples Core
Placement are added for each virtual core mapping, and
all temporal properties for that execution are updated. The
end result is that the execution is related to an Execution
Placement tuple, which is composed of all core mappings
with their time intervals defined according to process execution. Overall performance metrics can be added as properties
to the Execution tuple.
As proof of concept, we devised a scenario where a
benchmark from the NAS Parallel Benchmarks (NPB) suite is
executed. The scenario consists on executing eight processes
of the NPB EP (Embarrassingly Parallel) Class C benchmark
in two physical machines and four VMs, so that each VM
deploys two processes and each physical machine instantiates
two VMs. We chose Xen 4.1.1 as the hypervisor, and Ubuntu
Linux 10.04 64 bits as OS. Since the focus is not on actual
performance, we omit the hardware specification.
Fig. 5: Activity diagram for provisioning and execution.
Table I presents the adapted contents —altered for
readability— of some of the tables in the proposed model
after execution. Physical cores {1 − 8} correspond to the
first physical machine pm1 , while {9 − 16} correspond to
the second physical machine pm2 . The Physical Core
Groups section shows that six groups were created for
the GroupAllocate multi-association. From that entity, it
is possible to infer that virtual cores {1, 2} and {3, 4} —
which correspond to vm1 and vm2 , respectively— have been
allocated to dedicated, non-overlapping physical core groups
of pm1 . Virtual cores {5, 6} and {7, 8} —belonging to vm3
and vm4 , respectively— were all associated to a single group
containing all the physical cores of pm2 , which is the default
mapping. The table also reveals that the allocation of the
cores of vm4 was changed at a later time. The Virtual
Machine table indicates that all VMs belong to the same
execution and are deploying two SPMD processes each. Finally, the Execution Placement table contains a single
entry for that execution, with the throughput attribute as final
performance measurement. The temporal attribute contains all
time intervals where at least one VM was executing, and only
shows one time interval because of merging. The contents of
VM Placement —one tuple per VM— is omitted due to
space, but each tuple should hold a temporal value with the
union of time intervals when the processes were active.
This example shows that the proposed model can capture
all the details of the representation problem with desired
granularity. While it does not feature competing workloads,
the data should reveal the presence (but not the effect) of
contention by querying for resources at specific time intervals.
277
253
TABLE I: Adapted tuples from the model after execution.
Phy. Core Groups
gid
pcore
1
1
1
2
2
3
2
4
3
5
3
6
4
7
4
8
5
9
5
10
5
11
5
12
5
13
5
14
5
15
5
16
6
9
6
10
7
11
7
12
exid
1
vcore
1
2
3
4
5
6
7
8
7
8
vmid
1
2
3
4
metrics for each VM may also be added to the model.
Finally, we argue that this proposal can be reused for a
scheduling solution based on performance models working at
the level of resource provisioning. That scheduler would first
generate execution placement candidates for a given request
—which are guaranteed to be feasible by model constraints—
and then select an initial VM placement for application
provisioning. At runtime, the scheduler may use adaptive
strategies, such as changing core affinities and migrating VMs,
possibly taking into account server consolidation and average
application performance.
Core Placement
vmid
gid
temporal
1
1
[0, 16462]
1
2
[0, 16462]
2
3
[50, 16367]
2
4
[50, 16367]
3
5
[64, 16368]
3
5
[64, 16368]
4
5
[82, 10455]
4
5
[82, 10455]
4
6
[10455, 16371]
4
7
[10455, 16371]
Virtual Machine
exid
processes
1
{ SPMD, SPMD
1
{ SPMD, SPMD
1
{ SPMD, SPMD
1
{ SPMD, SPMD
Execution Placement
application
throughput
NPB EP C
11.80
R EFERENCES
}
}
}
}
[1] M. Armbrust, A. Fox, R. Griffith, A. D. Joseph, R. Katz, A. Konwinski,
G. Lee, D. Patterson, A. Rabkin, I. Stoica, and M. Zaharia, “A view of
cloud computing,” Commun. ACM, vol. 53, pp. 50–58, April 2010.
[2] “Magellan Final Report,” U.S. Department of Energy (DOE) Office of
Advanced Scientific Computing Research, Tech. Rep., Dec. 2011.
[3] S. K. Garg, S. K. Gopalaiyengar, and R. Buyya, “Sla-based resource provisioning for heterogeneous workloads in a virtualized cloud datacenter.”
in ICA3PP (1), ser. Lecture Notes in Computer Science, Y. Xiang et al.,
Eds., vol. 7016. Springer, 2011, pp. 371–384.
[4] J. Sonnek and A. Chandra, “Virtual putty: Reshaping the physical
footprint of virtual machines,” in Proceedings of the 2009 conference
on Hot topics in cloud computing, ser. HotCloud’09. Berkeley, CA,
USA: USENIX Association, 2009.
[5] R. N. Calheiros, R. Ranjan, and R. Buyya, “Virtual machine provisioning
based on analytical performance and qos in cloud computing environments,” in Proceedings of the 2011 Intl. Conf. on Parallel Processing.
Washington, DC, USA: IEEE Computer Society, 2011, pp. 295–304.
[6] J. Schad, J. Dittrich, and J.-A. Quiané-Ruiz, “Runtime measurements
in the cloud: observing, analyzing, and reducing variance,” Proc. VLDB
Endow., vol. 3, pp. 460–471, September 2010.
[7] P. Bientinesi, R. Iakymchuk, and J. Napper, HPC on Competitive Cloud
Resources. Springer, 2010, pp. 493–516.
[8] J. Ekanayake and G. Fox, “High Performance Paralllel Computing with
Clouds and Cloud technologies,” Indiana University, Tech. Rep., 2009.
[9] I. M. Llorente, R. S. Montero, B. Sotomayor, D. Breitgand, A. Maraschini, E. Levy, and B. Rochwerger, On the Management of Virtual
Machines for Cloud Infrastructures, ser. Wiley Series on Parallel and
Distributed Computing. Wiley Publishing, 2011, pp. 157–191.
[10] P. A. L. Rego, E. F. Coutinho, D. G. Gomes, and J. N. de Souza, “Faircpu: Architecture for allocation of virtual machines using processing
features,” in Proceedings of the 2011 Fourth IEEE International Conference on Utility and Cloud Computing, ser. UCC ’11. Washington,
DC, USA: IEEE Computer Society, 2011, pp. 371–376.
[11] A. V. Do, J. Chen, C. Wang, Y. C. Lee, A. Zomaya, and B. B. Zhou,
“Profiling applications for virtual machine placement in clouds,” in
Cloud Computing (CLOUD), 2011 IEEE International Conference on,
july 2011, pp. 660 –667.
[12] T. Wood, P. Shenoy, A. Venkataramani, and M. Yousif, “Black-box and
gray-box strategies for virtual machine migration,” in Proceedings of the
4th USENIX conference on Networked systems design & implementation.
Berkeley, CA, USA: USENIX Association, 2007, pp. 17–17.
[13] N. Bobroff, A. Kochut, and K. Beaty, “Dynamic placement of virtual
machines for managing sla violations,” in Integrated Network Management, 2007. IM ’07. 10th IFIP/IEEE International Symposium on, 21
2007-yearly 25 2007, pp. 119 –128.
[14] P. Mehrotra, J. Djomehri, S. Heistand, R. Hood, H. Jin, A. Lazanoff,
S. Saini, and R. Biswas, “Performance evaluation of amazon ec2 for
nasa hpc applications,” in Proceedings of the 3rd workshop on Scientific
Cloud Computing Date, ser. ScienceCloud ’12. New York, NY, USA:
ACM, 2012, pp. 41–50.
[15] G. Mc Evoy and B. Schulze, “Understanding scheduling implications for
scientific applications in clouds,” in Proceedings of the 9th International
Workshop on Middleware for Grids, Clouds and e-Science, ser. MGC
’11. New York, NY, USA: ACM, 2011, pp. 3:1–3:6.
[16] C. Parent, S. Spaccapietra, and E. Zimányi, Conceptual Modeling for
Traditional and Spatio-Temporal Applications: The MADS Approach.
Secaucus, NJ, USA: Springer-Verlag New York, Inc., 2006.
temporal
[0, 16371]
V. C ONCLUSION AND F UTURE W ORK
This ongoing work proposes a temporal conceptual model
for the representation of VM allocation. The allocation of a
given VM can change over time, either by configuration of
core mappings in the virtualization solution or by performing
live migration to another host.
We show that the model captures the characteristics of the
distribution of VMs, in particular of the allocation of CPU
processing, by specifying the virtual core mappings throughout
time. MADS was used to produce the temporal representation
of resource mapping, also to express model constraints for feasible allocations. The proof of concept shows how the model
can be used to register the placement of parallel applications
in virtualized environments, We expect that mining a database
with this model can reveal the effects of resource contention
and inter-process communication by correlating usage with
observed performance, thus we consider this a step towards
studying the performance of HPC applications in virtualized
environments.
As future work, we first plan to improve the conceptual
model to include temporal data on performance of the applications where applicable. Also, queries that provide details on
each execution placement need to be defined. In that context,
we plan to formulate a series of scenarios that can register
and evaluate the execution of a number of concurrent applications, with different execution placements, seeking to establish
correlations of characteristics of execution placements with
perceived performance. These experiments can also serve to
study the sources of resource contention in virtualized environments with applications that exhibit different communication
patterns. In that context, the model should provide abstractions
to represent features such as estimated communication-tocomputation ratio and message sizes. Resource consumption
278
254