1 An Efficient Cloud Market Mechanism for Computing Jobs with Soft Deadlines Ruiting Zhou, Zongpeng Li, Chuan Wu, Zhiyi Huang Abstract—This work studies the cloud market for computing jobs with completion deadlines, and designs efficient online auctions for cloud resource provisioning. A cloud user bids for future cloud resources to execute its job. Each bid includes (a) a utility, reflecting the amount that the user is willing to pay for executing its job, and (b) a soft deadline, specifying the preferred finish time of the job, as well as a penalty function that characterizes the cost of violating the deadline. We target cloud job auctions that executes in an online fashion, runs in polynomial time, provides truthfulness guarantee, and achieves optimal social welfare for the cloud ecosystem. Towards these goals, we leverage the following classic and new auction design techniques. First, we adapt the posted pricing auction framework for eliciting truthful online bids. Second, we address the challenge posed by soft deadline constraints through a new technique of compact exponential-size LPs coupled with dual separation oracles. Third, we develop efficient social welfare approximation algorithms using the classic primal-dual framework based on both LP duals and Fenchel duals. Empirical studies driven by real-world traces verify the efficacy of our online auction design. Index Terms—Cloud computing, Auction Mechanism, Online Algorithm. I. INTRODUCTION Cloud computing has emerged as a new computing paradigm that offers users rapid on-demand access to resources such as CPU, RAM and disk storage, with minimal management overhead. In the past decade, two types of cloud platforms blossomed on the Internet, including (i) largescale Internet data centers, exemplified by Amazon EC2 [1], Microsoft Azure and Linode [2], [3], which organize a shared resource pool for serving their users; and (ii) co-location data centers, often found in metropolitan areas, where smaller clouds from different users are physically co-located, jointly managed and serviced by the co-location [4]. Virtualization technologies help cloud providers pack their resources into different types of virtual machine (VM), for allocation to cloud users. For example, Amazon EC2 [1] currently offers 23 types of different VM types in 7 categories. Each type of VM has its focus and forte, and a large computing R. Zhou is with the University of Calgary, Calgary, AB, Canada (e-mail: [email protected]). Z. Li is with the University of Calgary, Calgary, AB, Canada, and also with the School of Computer and Collaborative Innovation Center of Geospatial Technology, Wuhan University, Wuhan, China (e-mail: zongpeng@ ucalgary.ca). C. Wu is with the University of Hong Kong, Kowloon, Hong Kong (e-mail: [email protected]). Z. Huang is with the University of Hong Kong, Kowloon, Hong Kong (e-mail: [email protected]). This work was supported in part by NSERC, Wedge Networks, and HKRGC (grant #: HKU 17202115E, HKU 718513, HKU 17204715, HKU 17225516, C7036-15G (CRF)). job often requires cooperation among multiple VM instances. For example, social games [5] and enterprise applications [6] are often composed of a front-end web server tier, a load balancing tier and a back-end data storage tier, each suited for execution on a VM that is abundant in a particular type of resource: bandwidth, CPU, or storage. Cloud computing jobs can be categorized into two types, depending on whether their computing need is elastic or not. Cloud jobs such as large scale web servers utilize cloud service as a utility, and require the rented VMs to be always active, with possible dynamic size scaling. These jobs are similar to the power users in a power grid who demands always-on power supply. Other jobs such as big data analytics and Google crawling data processing often have a batch processing nature. They require a certain computing job to be completed without demanding always-on VM service, and may tolerate a certain level of delay in the job completion. These users are similar to the energy users in a power grid who needs to draw a fixed quantity of energy for powering a given job, but in a flexible time window. Existing market mechanisms for cloud computing, particularly the auction type mechanisms, have been implicitly targeting the first type of non-elastic cloud jobs. In such oneround [7] and online [8] cloud resource auctions, once a bid is accepted, the service time window of the corresponding VMs is fixed, i.e. either in the current round [8] or between the start and finish times prescribed in the bid [9]. Such auction algorithms do not need to consider the scheduling of accepted jobs. In sharp contrast, a well designed market mechanism for the second type of elastic jobs must pay close attention to not only whether to accept a bid, but when to schedule its execution based on its deadline information. For example, consider a cloud user who bids for a VM bundle tailored for human genome analysis. Its job can be processed within 3 hours if the specified VM bundle is provisioned; however, as long as the computing result is available within the next 24 hours, the user is happy. This leaves ample space for job scheduling in the temporal domain, which a well-designed auction algorithm should judiciously exploit to maximize resource utilization and social efficiency — for example, scheduling a job within its tolerance window to time slots with relatively low demand. This work generalizes existing auction design in the cloud market by proposing online auctions that explicitly handle jobs with prescribed deadlines. We further allow a cloud user to express soft deadlines, described by both a preferred job completion time, coupled with a penalty function that encodes how much penalty is associated with different degrees of dead- 2 line violation. Compared with simple market mechanisms such as fixed pricing, a well-designed auction provides automatic price discovery, promptly adapts prices with the fluctuation of supply and demand, and allocates cloud resources to jobs who value them the most, maximizing the overall “happiness” of everyone in the cloud ecosystem. We simultaneously target the following goals in our cloud auction design. First, we require the cloud auction to be computationally efficient and executes in polynomial time. Second, the auction should be truthful, so that bidding true job valuation is the dominant strategy for a cloud user. Third, the auction should maximize the social welfare of everyone in the system including both the cloud provider and the cloud users. Such cloud auction design is faced with a number of challenges. First, truthfulness is a rather strong property that comes only with a pair of carefully prepared VM allocation and payment algorithms that work in concert with each other. Furthermore, even if the cloud users can be assumed to be altruistic and truthful bids are given for free, the winner determination problem for social welfare maximization is an integer linear program (ILP) that is NP-hard to solve. A new challenge unique to this work is the non-traditional type of soft deadline constraints, which is hard to model and handle with traditional LP formulation and algorithm design. Last but not least, we require the auction to be online, immediately making a decision upon the arrival of each bid, without knowing future bids in the market, yet still guaranteeing near-optimal decision making as compared to the offline optimum. We first consider a basic setting where resources in the cloud are free of cost up to a known capacity limit, and that the soft deadline can be expressed by enumerating a few hard deadline options and their corresponding bidding prices. We first present a natural ILP formulation of the social welfare maximization problem. While polynomial in size, this ILP involves both conventional constraints (capacity limits) and unconventional constraints (job deadlines). The latter further lead to unconventional dual variables that are hard to interpret and update in a primal-dual algorithm framework we will leverage. We convert the natural ILP into a compact-exponential ILP that has a compact formulation of conventional constraints only, at the price of involving an exponential number of variables. We apply the posted pricing primal-dual framework to the compact-exponential ILP for online social welfare maximization. Although the dual has an exponential number of constraints, we show fast dual oracles that can quickly update the dual variables, which are interpreted as unit cost of cloud resources in different time slots. We maintain carefully estimated resource costs based on recently designed exponential cost functions [10]. Upon receiving a bid, we compare the bidding price with the estimated cost of the bid. If the bidding price is higher, the bid is accepted and dual variables are updated; otherwise the bid is rejected. The posted pricing framework charges winning jobs an estimated cost that is independent from the bidding price, and is truthful [11]. We conduct theoretical analysis on the competitive ratio and prove its upper-bound. We proceed to generalize our cloud auction design by addressing two practical concerns. First, we model the cost of resource provisioning in data centers, using a convex cost function that characterizes server cost with Dynamic Voltage Frequency Scaling [12]. Second, we consider the general form of a soft deadline, specified by (i) a preferred deadline and (ii) a non-decreasing penalty function for deadline violation. The new social welfare maximization problem is an integer convex program. We resort to a new primal-dual solution framework for well-structured convex programs based on Fenchel dual [13], and adapt our posted pricing auction framework from the previous scenario to this general setting. In the rest of the paper, we discuss related work in Sec. II, and introduce the system model in Sec. III. Design and analysis of the online cloud auctions are presented in Sec. IV and Sec. V. Sec. VI presents simulation studies, and Sec. VII concludes the paper. II. P REVIOUS R ESEARCH Market mechanism design for cloud computing, particularly auction mechanisms for cloud resource trading, has attracted substantial interest from the research community, with a large number of VM auctions spawned in the past few years [7]–[9], [14]–[16]. The earliest VM auctions are simple in that they are oneround auctions, and assume that the cloud provisions a single type of VM, or that VM configurations are equivalent up to linear scaling [14]. They also assume the scenario of static VM provisioning, where the number and type of VMs to be sold are predetermined prior to the auction start [15]. Dynamic VM provisioning, in which the cloud provider makes decision on which VMs to assemble and how many based on demand learned from user bid during the auction, has been studied in the past two years [7]–[9]. Zhang et al. design a randomized auction for dynamic resource provisioning in cloud computing based on a convex decomposition technique, which is truthful and guarantees a small approximation ratio in social welfare [7]. Shi et al. further study dynamic resource provisioning where cloud users are subject to budget constraints, and design online auctions where decision making is coupled in the time domain due to fixed user budgets [8]. Online cloud auctions appear later than their one-round counterparts. Zhang et al. is among the first to study online cloud auction design, but they assume all VMs are of a uniform type [16]. The work of Shi et al. [8] designs online auctions, but does not consider the temporal correlation in decision making due to jobs spanning multiple time slots. A recent work of Zhang et al [9] study online cloud auctions where a user bids into a fixed time window for job execution; hence the scheduling dimension is non-present in their solution space. There have been recent studies on mechanism deign for batch jobs with deadlines. Lucier et al. study two scheduling algorithms for jobs with deadlines in cloud computing clusters [17]. They analyze the competitive ratio for non-committed scheduling, which does not require to finish executing a job that has started execution. They do not provide any performance guarantee on the competitive ratio for committed scheduling. Navendu et al. design a truthful allocation and pricing mechanism for computing jobs with deadlines, but 3 restrict attention to the offline setting [18]. Azar et al. construct an online mechanisms for preemptive scheduling with deadlines [19]. Their mechanism is truthful and achieves a constant competitive ratio. All of those work consider only one fixed deadline for each job, and fail to model the server’s operation cost. Compared with existing literature on cloud auctions, this work is the first to design cloud auctions that explicitly consider job elasticity and job execution deadlines, which are important for practical applications to batch processing jobs. Accordingly, we propose the compact-exponential optimization technique that can effectively handle the new job deadline constraints in social welfare maximization for the cloud. The online primal dual method (see [20] for a detailed survey) is a power algorithmic technique that that has witnessed broad applications, such as solving the ski rental problem, maximizing revenue in ad-auctions, and solving the general packing problem. The original primal dual framework works on linear programs, and is not used to solve problems modelled by convex programs in our work. Rather recently, new techniques were introduced to help apply the primal dual framework for algorithm design to convex programs. Blum et al. study online combinatorial auctions with production costs using the online primal dual framework [21]. They presents algorithms for various cost functions. Huang et al. further investigate the same problem and propose mechanisms with improved competitive ratio [11]. All those work don’t consider the scheduling of jobs, and they cannot handle VM departures and resource recycling. III. S YSTEM M ODEL AND P RELIMINARIES We consider a cloud data center hosting a pool of K types of resource, including CPU, RAM and disk storage that can be dynamically assembled into different types of VMs. Let [X] denote the integer set {1, 2, . . . , X}. There are a total ck unit of type-k resource in this cloud. The cloud service provider acts as the auctioneer to lease VMs to cloud users through an auction. User’s bid arrives randomly in a large time span 1, 2, . . . , T . Note that multiple bids can arrive simultaneously, and would be ordered randomly. There are I users participating in the auction, and each user requests multiple types of VM, and specifies in its bid: (i) rik , the total amount of type-k resource, and (ii) wi , the number of slots required to finish the job by the designated VMs. Job execution doesn’t need to be continuous. A user i’s job can be executed at any time slot as long as the total execution time meets wi before the deadline. We consider two soft deadline models in this work: a basic model with alternative deadlines and a general model with penalty function and server operation cost. A. Jobs with Alternative Deadlines We first consider a basic scenario where each user submits J optional bids to express disjunctive deadline options. A bid from user i consists of a list of desired types of resource rik , 8k; the number of requested slots wi , and deadlines for job completion dij , 8j, each with a corresponding bidding price bij . We use Bi to denote the bidding language of user i’s bids submitted at time ti : Bi = {ti , {rik }k2[K] , wi , {dij , bij }j2[J] }. We adopt the XOR bidding rule that assumes a user can win at most one bid among its J optional bids [7]. Upon the arrival of each bid, the cloud provider decides immediately whether to accept it, and if so, which deadline to choose and how to schedule the job. A binary xij equals 1 if user i’s jth bid wins, and 0 otherwise. Let another binary variable yi (t) encode the scheduling of user i’s job: yi (t) = 1 if user i’s job is scheduled to run at time t, and 0 otherwise. The cloud provider also calculates the payment pi for each winner i. Let vij be the true valuation of user i’s jth bid, then the utility of that bid with is uij (bij ) = vij pi if xij = 1, and is 0 if xij = 0. In practice, user are assumed to be selfish with a natural goal to maximize their own utilities; they may lie about their true valuations in the hope of a higher utility. The cloud provider instead pursues highest social welfare possible to make everyone in the cloud system “happy”. Thus, it is important for the cloud provider to elicit truthful bids. Definition 1. (Truthful Auction): A cloud auction is truthful if the dominant strategy for each user is to report its true valuation, which always maximizes its utility: for all bij 6= vij , uij (vij ) uij (bij ). Definition 2. (Social Welfare): The social welfare in the cloud market with alternative deadlines is the aggregate user utility P P P v x p i2[I] P j2[J] ij ij i2[I] i plus the cloud provider’s utility i2[I] pi . P Payments P cancel themselves, and the social welfare becomes i2[I] j2[J] vij xij . B. Jobs with Penalty Function and Operation Cost We further consider a more general model where each user submits a single preferred deadline di , with a penalty function gi (⌧i ) defined over deadline violation ⌧i : gi (⌧i ) = ⇢ gci (⌧i ), + 1, if ⌧ 2 [0, T otherwise di ] (1) where di + ⌧i is the job completion time; bi gi (⌧i ) is the bidding price, decreasing with job completion time; gci (·) is a nondecreasing function and gci (0) = 0. User i’s bid with this model is: Bi = {ti , {rik }k2[K] , wi , di , bi , gi (⌧i )}. Existing studies on cloud auction design often ignore the server operation cost of the cloud provider. It is natural to include server cost in the computation of social welfare, albeit the fact that it makes social welfare optimization substantially more challenging (from linear to non-linear integer programming). The operation cost in the cloud comprises mainly of power consumption for provisioning the virtual machines, increasing as the amount of resources used grows. Let zk (t) be the amount of type-k resource used at time t in the cloud, then the cost function of type-k resource is defined as: fk (zk (t)) = ( k zk (t) + 1, 1+ k , if zk (t) 2 [0, ck ] otherwise (2) Parameter k is the coefficient determined by the power consumption of each type of resource. Recent measurement studies suggest that the power consumption of memory, disk are significantly lower than that of CPU [22]. k 0 modulates the shape of the cost function, following the the operational model of physical servers in the cloud. For 4 example, Dynamic Voltage Frequency Scaling (DVFS) is a technique widely adopted in virtualization platforms, adjusting the frequency or voltage of CPUs to save power consumption [12]. k is roughly 2 if the voltage is proportional to the usage of CPU when DVFS is enabled, and equals 0 when DVFS in disabled [23]. The shape of RAM and disk cost function is different from that of CPU, with k 2 [0.5, 1] [22]. Similar to the notations in Sec. III-A, let a binary xi be an auction decision and pi be the payment. vi gi (⌧i ) is the true valuation of user i’s bid. The cloud provider’s utility equals the minus the operation cost, i.e., P aggregate P user payment P p f i2[I] i k2[K] t2[T ] k (zk (t)). The definitions of user i’s utility, truthful auction and social welfare are omitted here as similar ones can be found in Sec. III-A. Table I summaries notation for ease of reference. TABLE I: Summary of Notations I # of users [X] integer set {1, . . . , X} T # of time slots J # of bids per user f cost function f⇤ convex conjugate of f g penalty function ti user i’s arrival time pi user i’s payment ui user i’s utlity rik demand of type-k resource by user i wi # slots requested by user i ⌧i # slots that passes the deadline for user i dij (di ) deadline of user i’s jth (user i’s) bid bij (bi ) bidding price of user i’s jth (user i’s) bid vij (vi ) true valuation of user i’s jth (user i’s) bid xij (xi ) user i’s jth (user i’s) bid wins (1) or not (0) yi (t) whether or not to allocate user i’s job in slot t ck capacity of type-k resource pk (t) marginal price of type-k resource at time t zk (t) amount of allocated type-k resource at time t Uk (Lk ) maximum (minimum) value per unit of type-k resource per unit of time 1 max{2, (1 + k ) k } max{ ✓ckk k , ck (✓✓kk 1) ln( ✓k ⇢k IV. O NLINE AUCTION M ECHANISM FOR J OBS WITH A LTERNATIVE D EADLINES In this section, we focus on the scenario where each user’s job has J alternative deadlines. Sec. IV-A presents the social welfare maximization problem and the framework to handle such deadline problems. We design an online auction in Sec. IV-B and conduct theoretical analysis in Sec. IV-C. maximize X X bij xij (3) i2[I] j2[J] subject to: yi (t)t X j2[J] wi xij X X j2[J] dij xij , 8t 2 [T ], 8i 2 [I] : ti t, (3a) X yi (t), 8i 2 [I], rik yi (t) ck , 8k 2 [K[, 8t 2 [T ], (3c) X xij 1, 8i 2 [I], (3d) t2[T ]:ti t i2[I]:ti t j2[J] xij , yi (t) 2 {0, 1}, 8i 2 [I], 8t 2 [T ], 8j 2 [J]. (3b) (3e) simplified compact-exponential ILP with a packing structure, at the price of involving an exponential number of variables: subject to: XX XX bil xil (4) i2[I] l2⇣i rik xil ck , 8k 2 [K], 8t 2 [T ], (4a) X (4b) i2[I] l:t2l l2⇣i xil 1, 8i 2 [I], (4c) xil 2 {0, 1}, 8i 2 [I], 8l 2 ⇣i . Constraints (4a) and (4b) are equivalent to (3c) and (3d). ⇣i is the set of time schedules that satisfy constraints (3a) and (3b) for user i. The value of bil is based on schedule l, and equals the corresponding bij . We relax the integrality constraints of xil to xil 0 and formulate the dual problem. By introducing dual variables pk (t) and ui to constraints (4a) and (4b) respectively, the dual LP of the relaxed (4) is: minimize X ui + i2[i] subject to: ui bil X X pk (t), ui X X ck pk (t) (5) t2[T ] k2[K] k2[K] t2l Note that the following constraint is redundant, and is not P explicitly included in the ILP above: yi (t) j2[J] xij , 8i 2 [I], 8t 2 [T ]. Constraint (3a) ensures that a job is scheduled to run between its arrival time and deadline. Constraint (3b) guarantees that the number of allocated slots is sufficient for serving a successful bid. The capacity limit of each type of resource is expressed in constraint (3c), and the alternative deadlines are modelled with the XOR bidding rule by (3d). Even in the offline setting, ILP (3) without constraints (3a) and (3b) is still a NP-hard combinatorial optimization problem, equivalent to the classic knapsack problem. The challenge further escalates when we involve the jobs’ deadlines and pursue online decision making. To address these challenges, we resort to the primal-dual algorithm design technique. In preparation, we first design a new framework to handle the unconventional constraints for deadline modelling. More specifically, we reformulate the original ILP (3) into a )} competitive ratio of Aonline1 (Aonline2 ) ↵1 (↵2 ) maximize A. Social Welfare Maximization Problem Under the assumption of truthful bidding (bij = vij ), the social welfare maximization problem with alternative deadlines can be formulated into the following ILP: 0 Uk k k (1+ k )ck rik pk (t), 8i 2 [I], 8l 2 ⇣i , (5a) 0, 8i 2 [I], 8k 2 [K], 8t 2 [T ]. (5b) As we can observe, a feasible solution to ILP (4) has a corresponding feasible solution in ILP (3), and the optimal objective value of (4) is equal to that of (3). The number of variables in ILP (4) is exponential as the number of possible time schedules for user i is exponential in size. We next design an efficient primal-dual allocation scheme that only updates a polynomial number of variables, and can simultaneously solve optimization problems (3), (4) and (5). B. Online Auction Design In the auction algorithm, the cloud provider needs to deicide whether to accept a user i’s job and if so, how to schedule its job to meet its deadline. If user i’s jth bid with schedule 5 l is accepted, then let xij = 1, and update the variable yi (t) according to schedule l. To solve ILP (3), we adopt the primaldual technique to the compact-exponential ILP (4) and its dual (5). For each primal variable xil in (4), there is a dual constraint associated to it. Complementary slackness indicates the update of the primal variable is based on its dual constraint. xil is zero unless its associated dual constraint (5a) is tight. Because the dual variable ui 0, we let ui be the maximum of 0 and the right hand side (RHS) of (5a), ui = max{0, max{bil l2⇣i X X rik pk (t)}}. (6) t2l k2[K] Accordingly, the cloud provider accepts user i if ui > 0, and serves user i’s job according to the schedule that maximizes RHS of constraint (5a); if ui 0, the bid is rejected. Algorithm 1 A Primal-dual Online Auction Aonline1 Input: bidding language {Bi }, {ck } 1: Define function pk (zk (t)) according to (7); 2: Initialize xij = 0, yi (t) = 0, zk (t) = 0, ui = 0, pk (t) = 0, 8i 2 [I], 8j 2 [J], 8k 2 [K], 8t 2 [T ]; Let xil = 0, 8i 2 [I], 8l 2 ⇣i , by default; 3: Upon the arrival of the ith user 4: xij , {yi (t)}, pi , {pk (t)}, {zk (t)} {pk (t)}, {zk (t)} ; = pk (zk (t)) = Lk Acore1 Bi , {ck }, if 9j 2 [J], xij = 1 then Accept user i’s jth bid and allocated resources according to yi (t); Charge pi for user i; 7: else 8: Reject user i. 9: end if 5: 6: nential, we design a dual oracle that selects only a polynomial number of dual constraints. We fix a set of schedules Li with polynomial size through P the P dual koracle, and set ui = max{0, maxl2Li {bil t2l k2[K] ri pk (t)}}. Then xil is updated to 1 when ui > 0. The dual oracle works as follow. For each deadline dij of user i’s job, we select wi slots with the minimum price for t 2 [ti , dij ], and let lj be the corresponding schedule, and add lj to set Li . The schedule that maximizes user i’s utility is the one with the minimum price in set Li . We next discuss the update of the dual variable pk (t). Recall that pk (t) represents the marginal price per unit of type-k resource at time t. We define a new variable zk (t) as the amount of allocated type-k resource at time t, and let the marginal price be a function of zk (t). pk (t) is increasing with the growth of zk . Let Uk and Lk be the maximum and minimum values per unit of type-k resource per unit of time, respectively. pk (t) starts at Lk and exponentially increases when zk (t) is close to the capacity ck . It reaches Uk when zk (t) = ck because in this case, the cloud provider will never allocate any type-k resource to any user. In summary, pk (t) is defined as a function on zk (t) as following: Algorithm 2 A Scheduling Algorithm Acore1 Input: bidding language {Bi }, {ck }, {pk (t)}, {zk (t)} Output: xil , pi , {pk (t)}, {zk (t)} P k 1: c(t) = k2[K] ri pk (t), 8t 2 [T ]; // price per slot 2: for all j 2 [J] do 3: Select wi slots with minimum (c(t)) and zk (t) + rik ck , 8k 2 [K] P within [ti , dij ], save the schedule in lj ; 4: pij = t2lj c(t); uij = bij pij ; 5: end for 6: j ⇤ = arg maxj2[J] {uij }; 7: if uij ⇤ > 0 then 8: xij ⇤ = 1; yi (t) = 1, 8t 2 lj ⇤ , pi = pij ⇤ ; 9: xilj⇤ = 1; 10: ui = uij ⇤ ; zk (t) = zk (t) + rik , 8k 2 [K], t 2 lj ⇤ ; 11: pk (t) = pk (zk (t)), 8k 2 [K], t 2 lj ⇤ ; 12: end if 13: Return xij ⇤ , {yi (t)}, pi , {pk (t)}, {zk (t)} If we interpret dual variable pk (t) as marginal price per Pthe P unit of type-k resource at time t, then t2l k2[K] rik pk (t) is the total charge that user i should pay when its job is assigned according to schedule l. The RHS of (5a) becomes the utility of bid i with schedule l. Thus, the assignment of ui in (6) effectively maximizes user i’s utility. This is a key step towards achieving social welfare maximization and truthfulness. Note that although the calculation of ui seems to take exponential time as the size of dual constraint (5a) is expo- Where Uk mini2[I],j2]J] P ✓ Uk Lk ◆ zk (t) ck (7) b maxi2[I],j2]J] w ijrk and Lk i i . k w r k2[K] i i Aonline1 in Alg. 1 with the schedule algorithm Acore1 running for each user in Alg. 2 is the online auction. Aonline1 first defines the price function and initializes the primal and dual variables in lines 1-2. Upon the arrival of each user i, we select the bid j ⇤ with schedule lj ⇤ that maximizes user i’s utility through the dual oracle (lines 2-5). If user i obtains positive utility, primal variables xij ⇤ and yi (t) are updated according to schedule j ⇤ (line 8). We then increase the usage for different resources (zk (t)) and update the price (pk (t)) for t 2 lj ⇤ (lines 10-11). bij Slot 1 Slot 2 Slot 3 Slot 4 Slot 5 Z1(1)=2 p1(1)=1.2 Z1(2)=2 p1(2)=1.2 Z1(3)=2 p1(3)=1.2 Z1(4)=0 p1(4)=0 Z1(5)=0 p1(5)=0 C1=10 User i arrives at time 2 2 units bid 1: deadline 3, $6 2 slots bid 2: deadline 4, $4 { Accept bid 2. User i’s job is processed at time slots 3 and 4 Slot 1 Slot 2 Slot 3 Slot 4 Slot 5 Z1(1)=2 p1(1)=1.2 Z1(2)=2 p1(2)=1.2 Z1(3)=4 p1(3)=1.32 Z1(4)=2 p1(4)=1.148 Z1(5)=0 p1(5)=0 Fig. 1: An Example of the process in Aonline1 . We next use an example to illustrate the winner determination process in Aonline1 , as shown in Fig. 1. Suppose the online system spans 5 time slots. A cloud data center hosts only one type of resource and the capacity is 10, i.e., c1 = 10. Assume L1 = 1 and U1 = 2. Before the arrival of user i, assume the marginal price per unit of resource at time t is p1 (1) = p1 (2) = p1 (3) = 1.2; p1 (4) = p1 (5) = 0. The amount of allocated resource at time t is z1 (1) = z1 (2) = z1 (3) = 2; z1 (4) = z1 (5) = 0. User i arrives at time 2, 6 requiring 2 units of resource and 2 time slots to execute its job. It submits two optional bids: it is willing to pay $6 if its job is completed before time 3 or $4 if its job is finished before time 4. The bidding price of the user i can be expressed as Bi = {2, 2, 2, {3, $6}, {4, $4}}. Upon the arrival of the user i, Acore1 is executed to deicide whether to accept it and if so, how to schedule the job. The price per slot is calculated at line 1 in Acore1 and c(1) = c(2) = c(3) = 2.4; c(4) = c(5) = 0. For the first bid of user i, lines 3-4 in Acore1 compute the schedule, payment and utility of it: l1 = [2, 3], p11 = 4.8 and u11 = 1.2. For the second bid of user 1, l2 = [3, 4], p12 = 2.4 and u12 = 1.6. Now user i’s maximum utility is larger than 0, i.e., u12 > 0, primal and dual variables are updated accordingly at lines 8-11 in Acore1 . Here z1 (t) and p1 (t) at slots 3 and 4 are updated, i.e., z1 (3) = 4, z1 (4) = 2 4 2 and p1 (3) = 1 · 2 10 ⇡ 1.32, p1 (4) = 1 · 2 10 ⇡ 1.148. User i’s second bid is accepted and its job is processed at time slots 3 and 4. The cloud provider charges $2.4 for user i. This process is repeated until the last user’s job is handled. C. Theoretical Analysis i) Correctness, Polynomial Time, and Truthfulness. Theorem 1. Aonline1 computes a feasible solution to ILP (3), ILP (4) and LP (5) in polynomial time. Proof: (Correctness): Aonline1 outputs a feasible solution for ILP (3) because line 3 in Acore1 guarantees that the schedule lj for user i’s jth bid satisfies constraints (3a), (3b) and (3c). Constraint (3d) holds as only one bid per user can be accepted by Acore1 in line 6. Furthermore, the corresponding relation between xij and xil implies xil is a feasible solution for ILP (4). the dual problem (5), Acore1 P For P P assigns P 0 tok ui if bil k r p (t), and b il k2[K] t2l i k k2[K] t2l ri pk (t) to ui otherwise, ensuring the feasibility of Aonline1 . (Polynomial running time): Lines 1-2 can be executed in linear time for the initialization of the cost function, primal and dual variables. Upon the arrival of user i, Algorithm Acore1 first takes T steps to calculate the price of each slot. The for loop iterates J times to select the best slots for each bid. Line 3 in Alg. 2 takes O(T K) time to schedule the job and check the capacity limit. Line 4 can be done in O(1) steps. Thus, the running time of the for loop in Alg. 2 is O(JKT ). Then line 6 records the bid with the maximum utility in J steps. The body of the if statement (line 8-11) takes O(KT ) time to update the primal and dual variables and compute the payment. To sum up, the running time of Acore1 is O(JKT ). The last step of Aonline1 (lines 5-9) is to announce the auction decision, which can be done in constant time. In conclusion, Aonline1 runs in polynomial time (O(IJKT )). Theorem 2. The online auction Aonline1 is truthful. Proof: Our auction Aonline1 belongs to the family of posted pricing mechanisms [11]. Upon the arrival of user i, the payment that user i needs to pay to the cloud provider if it wins, depends only on the amount of resource that has been sold, and user i’s demand. It is independent of user i’s bidding price. Consequently, user i cannot improve its utility by lying about its bidding price as its utility equals its valuation minus the payment, i.e., uij = vij pi . Furthermore, Aonline1 always selects the schedule with the maximum utility among all possible schedules for user i. Hence, truthful bidding guarantees that each user obtains its maximum utility in Aonline1 . ii) Competitive Ratio. We next examine the competitive ratio of our online auction. The competitive ratio is the upper-bound ratio of the social welfare achieved by the optimal solution of ILP (3) to the social welfare achieved by our online auction Aonline1 . We first introduce the primal-dual analysis framework in Lemma 1, which guides the final proof of the competitive ratio. Let OP T1 and OP T2 denote the optimal objective values of ILP (3) and (4), respectively. We know that OP T1 = OP T2 . Let Pi and Di be the objective value of primal problem (4) and that of dual problem (5) returned by an algorithm after processing user i’s bids. Let P0 = D0 = 0 be the initial values. Then PI and DI are the final primal and dual objective values achieved by the algorithm. Lemma 1. If there exists a constant ↵1 1 such that Pi 1 Pi 1 (D D ) for all i, then the algorithm is ↵1 i i 1 ↵1 competitive in social welfare. Proof: When we sum up the inequalities for each i, we P P 1 have PI = Pi 1 ) Di 1 ) = ↵11 DI . i (Pi i (Di ↵1 According to weak duality [24], DI OP T2 , therefore 1 1 PI OP T = OP T . So we can conclude that the 2 1 ↵1 ↵1 algorithm is ↵1 competitive. Aonline1 guarantees P0 =D0 =0. We next define an Allocation-Price Relationship and show that if it holds for a given ↵1 , then the primal and dual objective values achieved by Aonline1 satisfy the inequality in Lemma 1. pik (t) denotes the price of type-k resource after handling user i. zki (t) is the amount of allocated type-k resource after processing i’s job. Definition 3. The Allocation-Price Relationship for Aonline1 with ↵1 1 is pik 1 (t)(zki (t) zki 1 (t)) ↵11 ck (pik (t) pik 1 (t)), 8i 2 [I], 8k 2 [K], 8t 2 l. Lemma 2. If the Allocation-Price Relationship holds for a 1 given ↵1 1, then Aonline1 guarantees Pi Pi 1 ↵1 (Di Di 1 ) for all i 2 [I]. Proof: If user i is rejected, then Pi Pi 1 = Di Di 1 = 0. In the following analysis, we assume that user i’s jth bid is accepted, and let l be the schedule of user i’s job. The increment of the primal objective value is: Pi Pi 1 = bil . Note that Aonline1 makes the constraint (5a) tight when bid bij with schedule l is accepted. Thus, bil = ui + P k2[K] P t2l pik 1 (t)(zki (t) zki 1 (t)). The increase of the dual objective value is: Di Di 1 = P P ui + k2[K] t2l ck (pik (t) pik 1 (t)). By summing up the Allocation-Price Relationship over all k 2 [K] and t 2 l, we can obtain: Pi Pi 1 ui + ↵11 (Di Di 1 ui ). Since ui 0 and ↵1 1, it is obvious that Pi Pi 1 ↵11 (Di Di 1 ). The Allocation-Price Relationship involves only the variables for type-k resource, we next try to find the corresponding ↵1,k for each resource k that satisfies the Allocation-Price Relationship. The value of the approximation ratio ↵1 is just the maximum value among all ↵1,k . In order to compute ↵1,k , 7 we assume that rik ⌧ ck , then zki (t) zki 1 (t) can be expressed as dzk (t). The derivative of the Allocation-Price Relationship under the above assumption is: Definition 4. The Differential Allocation-Price Relationship k for Aonline1 with ↵1,k 1 is pk (t)dzk (t) ↵c1,k dpk (t), 8i, k, t. Lemma 3. ↵1,k = ln and the marginal price defined in (7) satisfies the Differential Allocation-Price Relationship. Proof: The derivative of the marginal price function is: zk (t) 1 dpk (t) = p0k (zk (t)) = Lk (Uk /Lk ) ck ln(Uk /Lk ) ck . The Differential Allocation-Price Relationship is: reformulate the above convex optimization to the following compact-exponential convex problem: subject to: Lk Uk Lk ◆ zk (t) ck dzk (t) ) ↵1,k ck Lk ↵1,k Uk ln . Lk ✓ Uk Lk ◆ zk (t) ck 1 Uk ln dzk (t) ck Lk i2[I] l:t2l In this section, we present the online auction design for the general model that includes a penalty function and operation cost. We focus on the more challenging case of superlinear cost function with k > 0. The auction design for linear cost with k = 0 is similar and is omitted here. A. Social Welfare Maximization Problem Under the assumption of truthful bidding, the social welfare maximization problem in the general model is: i2[I] gi (⌧i )) X X fk (zk (t)) yi (t)t di + ⌧i , 8t 2 [T ], 8i 2 [I] : ti t, X wi xi yi (t), 8i 2 [I], i2[I]:ti t (8) t2[T ] k2[K] subject to: X (9a) X xil 2 {0, 1}, zk (t) (8a) (8b) t2[T ]:ti t yi (t)rik zk (t), 8k 2 [K[, 8t 2 [T ], (8c) ⌧i , zk (t) 0, xi , yi (t) 2 {0, 1}, 8i 2 [I],8t 2 [T ], 8k 2 [K]. (8d) Again, constraint yi (t) xi , 8i, t is redundant and is implied by the other constraints. Recall the definition of the cost function in (2) (fk (zk (t)) = +1 if zk (t) > ck ), constraint (8c) guarantees that the amount of allocated resource never exceeds its capacity. Let ⇣i be the set of time schedules that satisfy constraints (8a) and (8b) for user i, we adopt the same framework to (9b) xil 1, 8i 2 [I], 0, 8i 2 [I], 8l 2 ⇣i , 8k 2 [K], 8t 2 [T ]. (9c) We introduce dual variables pk (t) and ui to (9a) and (9b). The Fenchel dual [13] of the relaxed convex problem (9) is: X minimize ui + X X X X bil fk⇤ (pk (t)) (10) t2[T ] k2[K] k2[K] t2l V. O NLINE AUCTION D ESIGN FOR THE G ENERAL M ODEL WITH P ENALTY F UNCTION AND O PERATION C OST (bi xi rik xil zk (t), 8k 2 [K], 8t 2 [T ], l2⇣i subject to: ui Theorem 3. The online auction Aonline1 in Alg. 1 is ↵1 Uk competitive in social welfare with ↵1 = maxk2[K] ln L . k Proof: According to the proof in Lemma 3, ↵1 = Uk maxk2[K] ln L satisfies the Differential Allocation-Price Rek lationship. Under the assumption that dzk (t) = zki (t) zki 1 (t)) is much smaller than the capacity of type-k resource (ck ), we have dpk (t) = p0k (zk (t))dzk (t) = pik (t) pik 1 (t). As a result, we can obtain the Allocation-Price Relationship Uk holds for ↵1 = maxk2[K] ln L . Then, combining Lemma 1 k and Lemma 2 we finish the proof. X (9) t2[T ] k2[K] i2[i] Uk Therefore this lemma holds for ↵1,k = ln L . k maximize XX X X fk (zk (t)) bil xil i2[I] l2⇣i Uk Lk ✓ XX maximize rik pk (t), 8i 2 [I], 8l 2 ⇣i , (10a) 0, 8i 2 [I], 8k 2 [K], 8t 2 [T ]. (10b) pk (t), ui Where fk⇤ (pk (t)) is the convex conjugate [25] of the cost function fk (·), defined as: fk⇤ (pk (t)) = supzk (t) 0 {pk (t)zk (t) fk (zk (t))}. The explicit expression of the conjugate is as following: 8 ⇣ p (t) ⌘ 1+ k > k k > k < · 1 , ⇤ 1 + k k fk (pk (t)) = k > > : 1+ k ck pk (t) , k ck where zk0 (t) = ( Proof: fk⇤ (pk (t)) pk (t) k (1+ k ) = sup zk (t) 0 ( ) 1 k zk0 (t) ck zk0 (t) (11) > ck . pk (t)zk (t) pk (t)zk (t) k zk (t) 1+ k , 1, zk (t) 2 [0, ck ] zk (t) > ck We observe that pk (t)zk (t) 1 = 1 when zk (t) > ck , thus we only need to obtain the conjugate of f when zk (t) 2 [0, ck ], 1+ k Let k (zk (t)) = pk (t)zk (t) . The derivative k zk (t) of k (zk (t)) with respect to zk (t) is : k (zk (t)) 0 = pk (t) k (1 + k )zk (t) k . When we let k (zk0 (t))0 = 0, the local maximum happens at 1 k (t) the point zk0 (t) and zk0 (t) = ( kp(1+ ) k. k) Note that the domain of zk (t) is within the range [0, ck ], therefore the supremum of k (zk (t)) is zk0 (t) only if zk0 (t) 2 [0, ck ]. Otherwise, when zk0 (t) > ck , we can obtain that 0 k (zk (t)) > 0, which means k (zk (t)) monotonically increases with the increment of zk (t) and the supremum happens at zk (t) = ck . To sum up, we derive the conjugate of the cost function as shown in (11). B. Online Auction Design We adopt the same posted pricing primal-dual framework from Sec. IV to solve the convex problem (8). Similarly, the primal-dual technique is applied to its compact-exponential problem (9) and its dual (10): the primal variable xil remains zero unless its dual constraint (10a) becomes tight. The assignment of ui is the same as that in (6). 8 Although there are an exponential size of dual constraints in the computation of ui , we design a dual oracle based on dynamic programming to output a polynomial size of schedules, then only dual constraints associated to this set of schedules need to be considered. The basic idea of the dual oracle is as follows. We fix the completion time of user i’s job to be tc (tc 2 [ti + wi 1, T ]), and construct the best schedule lj with the minimum price in this case. The set that includes all such lj has polynomial size, and is the output of the dual oracle. The construction of lj is based on the dynamic programming method. The base case is the schedule l0 with t 2 [ti , ti + wi 1]. We move the P completion time one slot forward each time. Let c(t) = k2[K] rik pk (t) be the price of user i’s job running at time t. If the completion time tc passes the deadline di , thePcorresponding penalty is added to the price, i.e., c(t) = k2[K] rik pk (t)+g(tc di ). When the old competition time is replaced with the new one, we only need to compare the price of the old competition time and wi 1 slots before the old competition time. For example, if user i arrives at time 1 with wi = 4, then the basic case is l0 = {1, 2, 3, 4}. Assume that arg maxt2{1,2,3} c(t) = 2. We next fix the completion time to 5, the best schedule is then {1, 4, 3, 5} if c(4) < c(2) and {1, 2, 3, 5} otherwise. The process is repeated until the completion time reaches T . The marginal price pk (t) per unit of type-k resource at time t can be defined as the derivative of the cost function, i.e., fk 0 (zˆk (t)) if the overall demand of resource k at t (zˆk (t)) is known. But in the online setting, it is impossible for the cloud provider to acquire the complete knowledge of the system. The cloud provider predicts the final demand at future slots as ✓k (✓k > 1) times of the current demand at those slots if the predicted final demand is below the capacity, and set the marginal price to fk 0 (✓k zk (t)) where zk (t) is the amount of current allocated resource k at t. Let Uk0 be the maximum value per unit of type-k resource per unit of time. The marginal price grows exponentially when the predicted demand is larger than the capacity, and reaches Uk0 if zk (t) = ck . More specifically, the marginal price function is defined as: pk (zk (t)) = 8 0 > < fk (✓k zk (t)), > : fk0 (ck )e⇢k (zk (t) ck ✓k ck ck ) ✓k , zk (t) > ✓k zk (t) (12) 1 with parameters ✓k = max{2, (1 + k ) k }, ✓k ✓k ⇢k = max{ ln( k, ck ck (✓k 1) Uk0 )}, k k (1 + k )ck where Uk0 maxi2[I] wbirk . i i The online auction Aonline2 for the general model is presented in Alg. 3. Upon the arrival of the ith user, Aonline2 calls Acore2 in Alg. 4 to make decision. Acore2 computes the best schedule for user i through the dual oracle (lines 1-10) to maximize its utility ui . If ui > 0, the corresponding primal and dual variables are updated in lines 14-17. C. Theoretical Analysis i) Correctness, Polynomial Time, and Truthfulness. Lemma 4. The running time of Acore2 is O(KT + T 2 ). Algorithm 3 A Primal-dual Online Auction Aonline2 Input: bidding language {Bi }, {ck }, { k , k } 1: Define cost function fk (zk (t)) according to (2); 2: Define function pk (zk (t)) according to (12); 3: Initialize xi = 0, yi (t) = 0, zk (t) = 0, ⌧i = 0, ui = 0, pk (t) = 0, 8i 2 [I], 8k 2 [K], 8t 2 [T ]; Let xil = 0, 8i 2 [I], 8l 2 ⇣i , by default; 4: Upon the arrival of the ith user 5: xi , {yi (t)}, pi , {pk (t)}, {zk (t)} {pk (t)}, {zk (t)} ; = Acore2 Bi , {ck }, if xi = 1 then Accept user i’s bid and allocated resources according to yi (t); Charge pi for user i; 8: else 9: Reject user i. 10: end if 6: 7: Algorithm 4 A Scheduling Algorithm Acore2 . Input: Bi , {ck }, {pk (t)}, {zk (t)} Output: xi , {yi (t)}, pi , {pk (t)}, {zk (t)} 1: Add slot t 2 [ti , T ] to set T if zk (t) + rik ck , 8k 2 [K]; 2: Let schedule l0 include the first wi slots (t1 , t2 , . . . , twi ) in T ; Define j = 1; 3: while wi + j |T | do 4: lj = lj 1 ; 5: Let tc is Pthe (wi + j)th slot in T ; 6: c(t) = k2[K] rik pk (t), 8t 2 {t1 , t2 , . . . , twi , tc }; 7: If tc > di , c(tc ) = c(tc ) + g(tc di ); 8: tm = arg maxt2{t1 ,...,twi 1 } c(t); 9: If c(twi ) < c(tm ), for schedule lj , replace the slot tm with twi and P save tc into twi ; 10: Pj = t2lj c(t); j = j + 1; 11: end while 12: j ⇤ = arg minj {Pj }; 13: if bi Pj ⇤ > 0 then 14: xi = 1; yi (t) = 1, 8t P 2 lj ⇤ ; xP ilj ⇤ = 1; 15: ui = bi Pj ⇤ ; pi = k2[K] t2lj⇤ rik pk (t); 16: zk (t) = zk (t) + rik , 8k 2 [K], t 2 lj ⇤ ; 17: pk (t) = pk (zk (t)), 8k 2 [K], t 2 lj ⇤ ; 18: end if 19: Return xi , {yi (t)}, pi , {pk (t)}, {zk (t)} Proof: Line 1 initializes a feasible slot set T in O(KT ) steps. Line 2 takes wi steps to define a schedule lo The while loop (lines 3-11) is to compute the best schedule lj if the completion time is fixed, will iterate at most T wi times. Within the while loop body, lines 4-7 takes O(wi + 1) steps to update c(t). The running time of finding the maximum price in line 8 is linear to wi . Lines 9-10 takes constant time for the comparison and addition. Thus, the while loop can be executed in O((T wi )wi ) steps. Line 12 can be done in O(T wi ) steps to find the schedule with the minimum price. The running time to execute the if body is O(KT ). In summary, the running time of Acore2 is O(KT + wi (T wi )) O(KT + T 2 ). Theorem 4. Aonline2 in Alg. 3 is a truthful auction that returns 9 feasible solutions for convex problems (8), (9) and (10) in polynomial running time. Proof: (Polynomial running time:) The running time of the initialization process in lines 1-3 is linear. By Lemma 4, Acore2 in line 5 processes each user in O(KT + T 2 ) time. The If statement in lines 6-10 can be done within constant time. Therefore, after handling the last user, the overall running time of Aonline2 is O(I(KT + T 2 )). (Correctness and Truthfulness:) We omit the proof here as similar proofs can be found in Theorem 1 and Theorem 2. ii) Competitive Ratio. The proof follows the same structure as that in Sec. IV-C. Let Pi and Di be the primal (9) and dual (10) objective values achieved by Aonline2 after handling user i’s request. By Lemma 1, Aonline2 is ↵2 -competitive in social welfare if there 1 is a constant a2 1 such that Pi Pi 1 Di 1 ) ↵2 (Di for all i. We next define the Allocation-Price Relationship for Aonline2 , and show that if the Allocation-Price Relationship 1 holds for a given ↵2 , then Pi Pi 1 Di 1 ) also ↵2 (Di holds. The last step is to define the differential version of the Allocation-Price Relationship and prove there exists a ↵2,k that satisfies this relationship. By setting ↵2 = maxk2[K] {↵2,k }, we can obtain the competitive ratio of Aonline2 . Definition 5. The Allocation-Price Relationship for Aonline2 with ↵2 1 is pik 1 (t) zki (t) zki 1 (t) fk (zki (t)) fk (zki 1 (t)) 1 ↵2 fk⇤ (pik (t)) fk⇤ (pik 1 (t)) Pi Pi 1 = bil fk0 (zk (t)) = 0 fk⇤ (zk (t)) fk (zki (t)) fk (zki 1 = ⇢ k (1 + + 1, 8⇣ < : ck , pk (t) ⌘ k (1 + k ) pk (t) = k (1 X X pik 1 (t) zki (t) zki 1 ( k (1 + 1 ⇣ ↵2,k k )(✓k zk (t)) pk (t) + k) k (1 k) 1 k 1 fk (zki (t)) = ui + k ✓k ✓k fk (zki 1 k 1 Since ui 0 and ↵1 1 (D D i i 1 ). ↵2 k (1 1 k k (1 k +1 + k )ck pk (t) > k (1 + k )ck k k e ck ✓k ⇢k (ck ) Uk0 . + + k )zk (t) k )✓k k k )dzk (t) k zk (t) 1 k dzk (t). (13) k 2 · 2 k) ·2 k k (4 · 2 = k 2 1 2 1 4 k (2 k 1) 4 k < ↵2,k 2 k 1 k2 = 1 k ✓k = ✓k (1 + ✓k 1 X X fk⇤ (pik (t)) Case 2: zk (t) > ui + 1 (Di ↵2 Di 1 k) 1 k < e, and 0, it is obvious that Pi 1 Definition 6. The Differential Allocation-Price Relationship for Aonline2 with ↵2,k 1 is: pk (t)dzk (t) fk0 (zk (t))dzk (t) 1 f ⇤ 0 (pk (t))dpk (t), 8i, 8k, 8t ↵2,k k 2 l. < e(1 + k) < ↵2,k . In this case, the marginal price zk (t) is: k (1 + k k )ck e ⇢k (zk (t) ck ✓k ) . fk0 (zk (t)))dzk (t) 1 ck ⇢k pk (t)dzk (t). ↵2,k (14) By Lemma 7, we can obtain pk (t) fk0 (zk (t)) pk (t) 1 k p (t) p (t), thus to prove (14), it is sufficient to k k 1+ k 1+ k prove: k Pi k) Note that dpk (t) = ⇢k pk (t)dzk (t), then the Differential Allocation-Price Relationship is: (pk (t) ui ). ck ✓k : pk (t) = fk⇤ (pik 1 (t)) By summing up the Allocation-Price Relationship for Aonline2 over all k 2 [K] and t 2 l, we can obtain: Pi k )ck < 1, then ✓k = (1 + (t)) . t2l k2[K] Pi k (1 k +1 The second equality holds because Aonline2 update the value of dual variables such that dual constraint becomes tight and rik = zki (t) zki 1 (t). Then the increase of the dual objective value is: Di , pk (t) } = 2, we can obtain ii) If t2l k2[K] Di , if zk (t) 2 [0, ck ] otherwise Cancelling the common term on both sides, (13) becomes 1 k +1 (✓kk 1) . i) If k 1, ✓k = max{2, (1 + ↵2,k k ✓ k2[K] t2l X X k + k ⌘ (t)) (t) 1 k k Recall that Uk0 is the maximum value per unit of resource k per unit of time. It is clear when the marginal price is larger than Uk0 , no bids can win. Thus, we may assume zk (t) ck in the rest of the proof, and fk0 (zk (t)) = k (1 + k )zk (t) k . Next, we divide our proof into two cases: Case 1: zk (t) ✓ckk : Because pk (t) = f 0 (✓k zk (t)) = k (1 + k k (1 + k )ckk , the Differential Allocationk )(✓k zk (t)) Price Relationship can be rewritten as: t2l k2[K] = ui + k )zk (t) When the amount of allocated type-k resource reaches the capacity, i.e., zk (t) = ck , according to the definition of marginal price in (12), , 8i, 8k, 8t 2 l. Lemma 5. If the Allocation-Price Relationship for Aonline2 holds with a given ↵2 1, then Aonline2 guarantees Pi 1 Pi 1 (D D ) for all i 2 [I]. i i 1 ↵2 Proof: If user i is rejected, then Pi Pi 1 = Di Di 1 = 0. In the next analysis, we assume that user i is accepted, and let l be the schedule of user i’s job. The increment of the primal objective value is: X X U0 Lemma 6. ↵2,k = max{4(1 + k ), 2(1+k k ) ln( (1+ k )c k )} k k k and the marginal price function defined in (12) satisfy the Differential Allocation-Price Relationship. Proof: We first write down the explicit expressions for the differentials of the cost function (2) and its convex conjugate (11): 1+ 1 k ck ⇢k pk (t)dzk (t) ) ⇢k ↵2,k ck (1 + pk (t)dzk (t) k By the value of ⇢k , either i) ⇢k = = ✓k ck k e ck k ck (1 + k) k e(1 + k) k ck (1 + k) ↵2,k . k) ↵2,k . 10 normalize ck to 1, the competitive ratio of Aonline2 is close to 6 with a small Uk0 and k , as demonstrated in the left figure. The right figure shows that if ck is a large number, the competitive ratio is determined by k and increases with the increment of k . ✓k Uk0 Or ii) ⇢k = ln( ) k ck (✓k 1) k (1 + k )ck 0 2 Uk ln( ) k ck k (1 + k )ck 2(1 + k ) Uk0 k = ln( ) k ck (1 + k ) k k (1 + k )ck k ck (1 + k) VI. P ERFORMANCE E VALUATION ↵2,k . In conclusion, we have finished the proof for both cases. Lemma 7. When zk (t) > ✓ckk , the marginal price pk (t) is larger than the marginal cost by a factor of at least 1 + k : pk (t) (1 + 0 k )fk (zk (t)). Proof: When zk (t) > ✓ckk , pk (t) = k (1+ So Lemma 7 is equivalent to verify e⇢k zk (t) zk (t) k (1 + k )e k )ck k e ck ✓k ⇢k (zk (t) ⇢k ck ✓k ) . (15) ckk We first show that the inequality (15) holds when zk (t) = ✓ckk . If zk (t) takes the value of ✓ckk , (15) becomes ✓kk 1+ k which is obviously true. Next, it suffices to show the left side of (15) is nondecreeing as zk (t) increases. Let L(zk (t)) denote the left hand of (15). The derivative of L(zk (t)) is L0 (zk (t)) = e⇢k zk (t) (⇢k zk (t) zk (t)1+ k k) . ✓k ck Because ⇢k 0 k ck k and zk (t) > ✓k , then ⇢k zk (t) 0 and the derivative L (zk (t)) is nonnegative. Consequently, the lemma follows. Theorem 5. The online auction Aonline2 in Alg. 3 is ↵2 competitive in social welfare with ↵2 = maxk2[K] ↵2,k . Proof: Because ↵2 is the maximum number among all ↵2,k , then Differential Allocation-Price Relationship also holds with ↵2 . We assume that dzk (t) = zki (t) zki 1 (t) is much smaller than the capacity of type-k resource (ck ), then fk (zki (t)) fk⇤ (pik (t)) fk (zki 1 (t)) = fk0 (zki fk⇤ (pik 1 (t)) = 1 (t))(zki (t) ⇤0 i 1 fk (pk (t))(pik (t) zki 1 (t)), pik 1 (t)). Therefore, the Allocation-Price Relationship holds with ↵2 . Combinng Lemma 1 and Lemma 5, we finish the proof. 14 30 25 Competitive Ratio Competitive Ratio 35 Uk’=3, ck =1 20 15 Uk’=50, ck =1000 10 8 6 0.6 10 5 0.6 12 0.4 0.2 βk 0 0.5 1 1.5 γ k 2 2.5 0.4 0.2 βk 0 0.5 1.5 1 γ 2 2.5 k Fig. 2: An illustration of the competitive ratio of Aonline2 (↵2 ) under different settings. of We plot the value of ↵2 in Fig. 2 when we vary the value 0 k , k , ck and Uk [22], [23]. We can observe that if we We evaluate our online auctions Aonline1 and Aonline2 through trace-driven simulation studies. We exploit the trace version 1 in Google Cluster Data [26], which contains the information for each job including the start time, execution duration, and resource demands (CPU and RAM). We translate each job into a bid, arriving sequentially in 18 hours. We assume that each user’s job consumes [1, 12] slots and each time slot is 5 minutes [26]. User’s job deadline is generated uniformly at random between its arrival time and the system end time. The bidding price of each job equals its overall resource demand times unit prices randomly picked in the range [Lk , Uk ]. By default, Lk = 1 and Uk0 = Uk = 50. The demand for CPU and RAM units is normalized so that the maximum capacity is 1. For the cost function, k is set within [0.4, 0.6] for CPU and within [0.005, 0.02] for RAM [22]. k is set within [1.7, 2.2] for CPU and within [0.5, 1] for RAM [23]. Performance of Aonline1 . We examine the performance of Aonline1 in terms of the competitive ratio, social welfare and user satisfaction. Fig. 3 shows the competitive ratio of Aonline1 with different number of users (I) and bids per user (J). The observed competitive ratio is much better than the theoretical bound and remains at a low level (< 2). It fluctuates with the increase of the number of users and sightly decreases when the number of bids per user grows. This is because when each user provides a larger number of optimal bids, Aonline1 is more likely to optimize the schedule of its job, leading to a better performance. In Aonline1 , the marginal price function is defined based on the real value of Uk and Lk . We vary the value of Uk /Lk , and use the estimated values of Uk as the input of Aonline1 , to examine the performance. As shown in Fig. 4, there is a downward trend as the value of Uk /Lk decreases, while there is no large difference with either underestimation and overestimation. The observation confirms the analysis in Theorem 3 that the value of Uk /Lk determines the competitive ratio. Underestimation is more desirable than the overestimation, as compared to that achieved by the real Uk (labelled by 100%). Overestimation makes the price rise more rapidly, filtering out users that are supposed to be accepted. We next study the social welfare achieved by Aonline1 in Fig. 5 and Fig. 6. The 3d figure in Fig. 5 plots the social welfare under different number of users and bids per user. Our online auction Aonline1 achieves a higher social welfare when there is larger number of users participating the auction. The change of bids per user doesn’t have major influence on the social welfare. When the number of users grows, the number of bids with larger bidding price also increases. As a result, Aonline returns a larger social welfare. The social welfare under different number of slots and Uk /Lk is illustrated in 2 1.6 1.61.6 1.6 8000 8000 80008000 8000 1.4 1.4 1.41.4 1.4 1.2 1.21.2 1.2 1.2 1 1 1 1 10 10 20 20 3030 4040 5050 60 1 10 30 30U40/L40 50 50 60 6060 101020 20 20 50 60 kU k /L k k 40 U /L30 0.5 0.5 0.5 40 50 70 70 80 80 90 90 100 100 0.5 40 60 50 0.5 40 6060 60 7070 80 9090 100 40 50 50Number 80 of Users of 40 50 60 Number 70 80Users 90 100 100 Number of Users Number of Users k Number of Users Ukk /L Uk /L k k 6000 6000 60006000 6000 4000 4000 40004000 4000 2000 2000 20002000 2000 0 0 0 120 0 120 0 120 120 80 10 10 10 8 10 801208080 8 6 88 40 10 6 4 4 66 8 40 80 40 2 4 40 Number ofUsers Users40 00 20 0 2 4 J4J 6 Number of J Number of Users 0 2 0 Number of Users 0 0 2 J J Number of Users 0 0 Provider Revenue Provider Revenue Provider Revenue Competitive Ratio 1.6 Social Welfare J=2 J=2 J=2 J=4 J=2 J=2J=4 J=4 J=6 J=4 J=4J=6 J=6 J=6 J=6 1 Competitive Ratio 1 111 Competitive Ratio 1.5 1.5 1.5 1.5 1.5 50% 50% 100% 50% 50% 50% 100% 150% 100% 100% 100% 150% 150% 150% 150% Social Welfare 1.8 1.8 1.81.8 1.8 Social Welfare SocialWelfare Welfare Social 2 2 22 2 Competitive Ratio Competitive Ratio Competitive Ratio 2 222 Competitive Ratio Competitive Ratio Competitive Ratio Competitive Ratio Ratio Competitive 11 1.8 Fig. 1: Competitive ratioratio of of Aonline1 Fig. Social welfare ofAA with Fig. ratio ofofonline1 AAonline1 Fig. 1: Competitive Aonline1 3: Social Social welfare with Fig. 2:Competitive Competitive ratio online1 online1 Fig. 5: welfare ofonline1 online1 Fig. 1: Competitive of Fig.Fig. 3: Social welfare of A withwith Fig. 2:2: ratio of of online1 Fig. 4:Competitive Competitive ratio A Fig. 3: ratio ofof A AA Fig. 1: 1: Competitive Competitive ratio ratio online1 3: Social welfare of Aof 2: Competitive ratio ofAof Aonline1 online1 Fig. online1 online1 Fig. Competitive ratio of A Fig. 3: Social welfare A with Fig. 2: Competitive ratio A online1 with different number of users and J. with different number of users and J. online1 number of different number of users and J. online1 with different U /L . with different U /L . different number of users and J. k k k k with different number of users and J. different number of users and J. with different U /L . with different different kUk /L k k .. with different users and J. withdifferent differentnumber numberofof different number of users and J. k . with number of users users and and J. J. with different number of users and J. 2 with differentUkU/L /L 2 k k k UUk /L/Lk =10 =10 U k /L k =10 k k UU /L/L=10 =30 =30 kUk k/L U k /L k =30 k kU k /L =10 UU /L =30 /L =50 k k kUk k/L U k /L k =50 k =50 k U k /L =30 U k /L k =50 k k U k /L k150 =50 Provider Revenue k Social Welfare Competitive Ratio 100 Social Welfare Percentage of Winners Percentage of Winners k PercentageofofWinners Winners Percentage Percentage of Winners k U k/L =10 U kk/L UU /L k=20 =20 k=20 9090 10090 k k/L U k/Lkk=20 90 U /L =30 80 90 80 /L =20 U /LU=30 5 /L k=10 UU/L =10 k =10 1.5 U k/L U k k k /L k/L =10 =30 4 U k/L k=30 UkU/L k =30 k k Uk /L =10 U kU /L =30 /L k =50 k U k/L k=50 kk =50 k 6000 U /L k U=50 k /L =30 6000 6000 U /L 3 k k 6000 Social Welfare Social SocialWelfare Welfare Competitive Ratio 110110 110 110 U=10 /L =10 UU /L/L=10 k k 100 k kk k 110 100 100 U /L =10 Social Welfare Social Welfare Social Welfare Social Welfare Social Welfare 7000 7000 7000 7000 7000 6000 6000 6000 6000 5000 5000 5000 6000 5000 4000 4000 4000 5000 4000 3000 3000 3000 4000 3000 2000 2000 2000 3000 2000 1000 1000 1000 2000 1000 50% 1.6 150 150 1501.4 U k /L U kk=10 /L k =10 U k /L k =10 100% UUkk/L /L =30 Ukkk=10 /L k =30 U k /L k =30 150% U k /L k =10 UUk /L =30 /L =50 U k /L k =50 k Ukk /L =50 k k U k /L k =50U k /L k =30 U k /L k =50 Fig. Competitive Users801: 90 50Number 60 of70 100 ratio of A Provider Revenue 1 J=2 U k/L k=50 k k U 80 150 k k/L k k =30 6000 4000 U k/Lkk=30 80 1.2 100 100 J=4 4000 4000 100 2 70 70 U /L =40 100 4000 U=40 /L kk=30 80 J=6 UU /L 70 k k k /L k =40 70 Ukk/Lkk=40 4000 0.5 50 1 100 2000 2000 50 1 60 7060 30 40 40 50 60 702000 90 U 100 ’=3 Uk’=5050 Uk10 ’=100 20 U k/L =50 60 /L =40 U k/LUk=50 2000 80 k 50 60 k k U k/Lkk=50 Uk /L k Number of Users 50 U k/L k=50 50 0 0 0 2000 0 50 0 60 40 50 60 70 80 90 100 0 50 40 50 60 70 80 90 100 0 120 120140 140160 160180 180200 200 12 14 0 16 0 18 70 20 90 50 1000 70 90 110 U k/L k=50 0 50 0 050 70 10 90 110 40 50 60 70 of 80 80 90 90 100 100 50 0 50 70 90 110 110 Users 120 160 180 40 50Number 60Number of 70 Users Number of Slots 200 90Number 110of Users 70 of90 90 110110 120 140 140 160 180 200 Number of Slots of Number Users 5050Number 70 90Users 110 50 Number of70 Users 5050 Number of Users Number of70Users 0 Number of Slots 2:Number 0Number ofonline1 0 Competitive Users Fig. Number of Users ratio Number of Users 50 70Users 90 110 50 of70 90 110 Number of Users with different number of users and J. with different Ukof/L k. ofwelfare Slots of A 4: Number Social Aonline1 Fig. 5: Percentage ofwinners winners Fig. Fig. 6: welfare withwith Number of Users Number Users Fig. Percentage AA Fig. 8: Competitive ratio Aonline2 with online1 online1 4: Social of Aofonline1 with Fig. 5:7:Percentage of of winners in in Ain online1 Fig. 6: Social welfare and of cloud provider’s online1 Fig. 6: Social welfare and cloud provider’s Fig. 4:4:Social ofof AAonline1 Fig. 5: Percentage of winners winners in AAonline1 online1Fig. Fig.different Social welfare with Fig. 5: Percentage of in 0 Fig. 6:Social Social welfare andcloud online1 with T welfare and Ukkk/L with different and different T and U 6:in welfare and with different I Iand k k /L different I A and Uwith with different I and U /L k /L kU/L k k revenue Icloud andprovider’s Uprovider’s k kU k online2 k /Lk k with revenue Ain I and online2 110 Uk /Lk different T and U /L with different I and U /L7000 120 140Number 160of Slots 180 200 40 k k Percentage of Winners Fig. 4: Social Fig. different 5: Percentage winners in Aonline1 different T andwelfare Ukk /Lkk of Aonline1 withwith I and of Ukk/L revenue withI Iand and /L kk Fig. 6:ininSocial welfare cloud revenue AA UkU/L online2 kprovider’s online2 k k U /L =10with and 6000 100 different T and Uk /Lk with different I and Uk /L k revenue in Aonline2 with I and Uk /Lk 50 of U U /L k=20UF.’=10 Li,U kand Lau,90 k k k k k Provider Revenue Social Welfare k Percentage of Winners Social Welfare Percentage of Winners Percentage of Winners Percentage of Winners Percentage of Winners Percentage of Winners Fig. 6. Both ’=10 U Shi,Zhang, L. Zhang, C. Wu, “An online auction 100 number of slots and the value of Uk /Lk [8] W. Shi, L. C. U Wu, Z. Li,Z. and F. Lau, “An online auction 100 the U F. ’=30 Lau, 4000 U resource .=30 framework for dynamic resource provisioning in80cloud computing,” in [8] W. Shi,for L.dynamic Zhang, C.Wu, Wu, Z.Li,Li, and “Anonline onlineauction framework provisioning in cloud computing,” in auction [8] W. Shi, L. Zhang, C. Z. and F. Lau, “An 100 influence the welfare. A is able to allocate more 100social U /L =30 online1 k k U ’=50 80 80 U ’=50 Proc. of ACM SIGMETRICS, 2014. 3000 Proc. of ACM SIGMETRICS, 2014. framework fordynamic dynamic resource provisioning cloud computing,” framework for resource provisioning in incloud computing,” in in jobs when 100 the 6000 L. Zhang, C. Wu, 150 Z. Li, and F. 70Lau, “An online auction [8] W.Zhang, Shi, 80 80 length of the system increases. Furthermore, [9] X. Z. SIGMETRICS, Huang, C. Z. Wu, Z.and Li,F.and Lau, “Online auctions in IaaS ULau, /L F. =40 [9] X. Zhang, Z. Huang, C. Wu, Li,2014. Proc. ACM SIGMETRICS, 2014. Proc. ofof ACM k k “Online auctions in IaaS 2000 60 60 framework for dynamic resource provisioning in cloud computing,” the bidding price rises with the increase of Uk /Lk , thus high welfare and C. profit maximization with server in Proc. of 60 costs,” clouds: welfare profit maximization with server costs,” in auctions Proc. of [9] X. Zhang, Z.Z.and Huang, Wu, Z.Z.Li, F.F. Lau, “Online auctions in in IaaS [9] X.clouds: Zhang, Huang, C. Wu, Li,and and Lau, “Online IaaS in 1000 60 40 80 60 U k/L k=50 100 2014. 4000 Proc. of ACM SIGMETRICS, ACM SIGMETRICS, 2015. 40 I=80 ACM SIGMETRICS, 2015. I=80 value bids lead to a higher social welfare. clouds: welfare in in Proc. of of clouds: welfareand andprofit profitmaximization maximizationwith withserver server costs,” Proc. 50 costs,” 0N. I=90 [10] J.2015. Naor, “Online primal-dual algorithms for covering [9] X.Buchbinder Zhang, Z.160 Huang, C. Wu, Z. Li, andalgorithms F. Lau, “Online auctions in 50 60 70 IaaS 80 90 I=90 Buchbinder J.and Naor, “Online for40covering 40 20 40 60 120 140 and 180 200 primal-dual ACM SIGMETRICS, ACM SIGMETRICS, 2015. I=80 I=80 20 User satisfaction which is measured by the percentage of [10] N. 50 2000 I=100 Number of Users and packing problems,” in ESA. Springer, 2005, pp. 689–701. I=100 clouds: welfare and profit maximization with server costs,” in Proc. of Number of Slots andN. packing problems,” inNaor, ESA.“Online Springer, 2005, pp.algorithms 689–701. I=90 [10] Buchbinder and J.J. primal-dual forfor covering I=90 [10] N. Buchbinder and Naor, “Online primal-dual algorithms covering 20 Z.ACM Huang and A. Kim, “Welfare maximization with production a winners is 20 demonstrated in Fig. 7. A higher fraction of users [11] [11] 40 0 Z. Huang andSIGMETRICS, A. Kim, “Welfare maximization with production costs:costs: a 2015. 0 I=80 I=100 and packing problems,” in Springer, 2005, pp.pp. 689–701. I=100 and packing problems,” inESA. ESA. Springer, 2005, 689–701. 0 0 180 120 120 130 130 140 140 150 150 160 160 170 170 180I=90 primal dual approach,” in Proc. of ACM-SIAM SODA, 2015. primal dual approach,” in Proc. of ACM-SIAM SODA, 2015. 60 80 100 60 80 100 [10] N. Buchbinder and J. Naor, “Online primal-dual algorithms for covering are accepted20 a smallNumber number ofofusers. [11] Huang and A. “Welfare maximization with costs: Fig.Z. Social welfare of Aonline1 with Fig. 5:production Percentage of awinners in 0 [11] Z.4: Huang andNumber A.Kim, Kim, “Welfare maximization with production costs: a Number Slots This is because the 0with of Slots of Users Number ofhttp://en.wikipedia.org/wiki/ Users Wikipedia, Dynamic frequency scaling, [12] [12] Wikipedia, Dynamic frequency http://en.wikipedia.org/wiki/ I=100 120 130 130 140 140 150 150 160 170 170 180 180 and dual packing problems,” inscaling, ESA. Springer, 2005, pp. 689–701. 120 160 primal approach,” ininProc. ofofACM-SIAM SODA, 2015. primal dual approach,” Proc. ACM-SIAM SODA, 2015. number ofFig. winners almost fixed due to the capacity limit. different T and U /L with different I and U /L Fig. 7:isPercentage of winners in A Dynamic frequency scaling. k k k k 7: Percentage of winners in A online2 Dynamic frequency scaling. online2 NumberofofSlots Slots Fig. 9:Wikipedia, Social welfare cloud provider’s revenue in production Aonline2 costs: a [11] Z. Huang and A.and Kim, “Welfare maximization with Number [12] Dynamic frequency scaling, http://en.wikipedia.org/wiki/ 0 [12] Wikipedia, Dynamic frequency scaling, http://en.wikipedia.org/wiki/ [13] N. R. Devanur, “Fisher markets and convex programs,” JACM, 2010. We also obverse the Uk160 /Lk170 doesn’t influence the [13] N.IDynamic R. Devanur, “Fisher markets convex programs,” JACM, 0frequency 120 130value 140 of 150 180 Fig.7:7:that Percentage winners in A A primal dual approach,” inand Proc. of ACM-SIAM SODA,2010. 2015. scaling. online2 with and U . Fig. Percentage ofof winners in Dynamic frequency scaling. online2 k [14] Q. Wang, K. Ren, and X. Meng, “When cloud meets ebay: Towards [14][13] Q. Wang, K. Ren, Dynamic and X. markets Meng, “When cloud programs,” meets ebay:JACM, Towards Number of Slots the percentage winner determination process is not affected N. R. Devanur, “Fisher and convex 2010. [12] Wikipedia, frequency scaling, http://en.wikipedia.org/wiki/ inasauction thetheauction design include posted pricing frameusedused in the design include the posted pricing frame[13] N. R. Devanur, “Fisher markets and convex programs,” JACM, 2010. [8] W. Sh effective pricing for cloud computing,” in Proc. of IEEE INFOCOM, effective pricing for cloud computing,” in Proc. of IEEE INFOCOM, 100 Fig.truthful 7: U Percentage of winners in LP Aonline2 Q. Wang, Ren, Towards Dynamic scaling. by the change of [14] Q.2012. Wang, K. K.frequency Ren,and andX. X.Meng, Meng,“When “Whencloud cloudmeets meetsebay: ebay: Towards framew work for online auctions, a new formulation and [14] k /L k .auctions, work for truthful online a new LP formulation and 2012. used in the auction design include the posted pricing frameused in the auction design include the posted pricing frame- [15] [15] effective pricing cloud computing,” of ofIEEE INFOCOM, [13] R. Devanur, “Fisher markets and in convex programs,” JACM, 2010. Proc. o effective pricing for cloud computing,” inProc. Proc. IEEE S.N.Zaman and D.for Grosu, “Combinatorial auction-based dynamic vm proZaman and80 D. Grosu, “Combinatorial auction-based dynamic vm INFOCOM, prosolution method for handling asand well S. solution handling soft soft deadline as well Performance of for A .auctions, work formethod truthful online a deadline new constraints, LP constraints, formulation 2012. online2 [14] Q. Wang, K. Ren, and X. Meng, “When cloud meets ebay: Towards visioning and allocation in clouds,” in Proc. of IEEE Cloud CloudCom, work for truthful online auctions, a new LP formulation and 2012. visioning and60allocation in clouds,” in Proc. of IEEE Cloud CloudCom, [9] X. Zha used in the auction design include the posted pricing frameas first approximation algorithms onofdual LP dual Fenchel [15] S. Zaman D. auction-based proas approximation algorithms based on and and We examine the competitive ratioLP Aonline2 .Fenchel We use effective pricing for “Combinatorial cloud computing,” in Proc.dynamic ofdynamic IEEEvmINFOCOM, solution method for handling softbased deadline constraints, as well [15] S.2011. Zamanand and D.Grosu, Grosu, “Combinatorial auction-based vm pro- clouds: 2011. solution method for handling soft deadline constraints, as well visioning and allocation in clouds,” in Proc. of IEEE Cloud CloudCom, work for Our truthful online aon new formulation and 2012. dual. method for auctions, handling deadline constraints H. Zhang, B. H. Jiang, Liu, A. Vasilakos, and J.“A Liu, “A framework dual. Our method for handling soft deadline constraints maymay [16] [16] H. Zhang, B. 40 Li, H.Li, Jiang, F. Liu, A. Vasilakos, andI=80 J.of Liu, framework visioning and allocation inF.clouds,” in Proc. IEEE Cloud CloudCom, ACM S CVX with the Gurobi Optimizer to soft solve theLP convex asapproximation approximation algorithms based LP dual andproblem Fenchel 2011. as algorithms based on problems LP dual and Fenchel for in cloud computing with heterogeneous [15] S. truthful Zaman and D.auctions Grosu, “Combinatorial auction-based dynamic vm[10] pro-N. Buc I=90 for truthful onlineonline auctions in cloud computing with heterogeneous user user 2011. be applicable to other auction design where deadline solution method for handling soft deadline constraints, as well be applicable to other auction design problems where deadline (8) exactly, and compute the competitive ratio by dividing 20 dual.Our Ourmethod methodfor forhandling handling soft soft deadline deadline constraints constraints may [16] H.demands,” Zhang, B. in Li, H. Jiang, F. Liu, A. Vasilakos, andofJ. IEEE Liu, “ACloud framework I=100 Proc. of IEEE INFOCOM, demands,” in Proc. of IEEE INFOCOM, 2013. visioning and allocation inLiu, clouds,” in2013. Proc. CloudCom,and pa dual. may [16] H. Zhang, B. Li, H. Jiang, F. A. Vasilakos, and J. Liu, “A framework is involved, for example, in demand response auctions forA. truthful online inN.cloud computing with heterogeneous user [11] Z. Hua isbeapproximation involved, forto example, inbydemand response auctions in ain a [17]Kansal, as algorithms based on welfare LP dual anddeadline Fenchel the optimal social welfare the social returned by Kansal, F. Zhao, J. N. Liu, Kothari, and A. Bhattacharya, “Virtual applicable other auction design problems where F. Zhao, J. auctions Liu, Kothari, and A. A.A. Bhattacharya, “Virtual 0 online for2011. truthful auctions in cloud computing with heterogeneous user besmart applicable to other auction design problems where deadline [17] A. demands,” in Proc. of130 IEEE INFOCOM, 2013. 120metering 140 150 160 170 180 smart grid. machine power and provisioning,” in Proc. of Liu, ACM SoCC, primal grid. machine power metering and provisioning,” in Proc. of ACM SoCC, dual. Our method for handling soft deadline constraints may Aisonline2 . However, CVX fails to solve in 24 hours even with [16] H. Zhang, B. Li, H. Jiang, F. Liu, A. Vasilakos, and J. “A framework demands,” in Proc. of IEEE INFOCOM, 2013. involved, for example, in demand response auctions in a [17] A.2010. Number of Slots Kansal, F. Zhao, J. Liu, N. Kothari, and A. A. Bhattacharya, “Virtual [12] Wikipe isa medium-size involved, forinput. example, inwedemand response auctions in a [17] 2010. for truthful online auctions in cloud computing with heterogeneous user A. Kansal, F. Zhao, J. Liu, N. Kothari, and A. A. Bhattacharya, “Virtual Thus, reduce the input size and only be applicable to other auction design problems where deadline smart grid. 7:A. Percentage ofprovisioning,” winners power metering andR. in Proc.provisioning of ACM SoCC, R EFERENCES Fig. 10:Beloglazov, Percentage of winners in“Power-aware Aonline2 Dynam R EFERENCES K. H.Fig. Kim, Beloglazov, and R. Buyya, provisioning [18] [18] K. machine H. Kim, A. “Power-aware demands,” in Proc. ofand IEEE INFOCOM, 2013. smart grid.10-20 users. Fig. machine power metering and Buyya, provisioning,” inonline2 Proc. of ACM SoCC, 2010. [13] N. R. D consider the response competitive ratio ofin a of of virtual machines for real-time cloud services,” Concurrency virtual machines for real-time cloud services,” Concurrency and and is involved, for example, 8in shows demand auctions [17] A. Kansal, F. Zhao, J. Liu, N. Kothari, and A. A. Bhattacharya, “Virtual 2010. Rtype, EFERENCES [18] K.Computation: H. Kim,Practice A. Practice Beloglazov, R. vol. Buyya, “Power-aware and and Experience, vol.no. 23,13, no.pp. 13,1491–1505, pp.provisioning 1491–1505, [14] Q. Wa [1] Amazon instance http://aws.amazon.com/ec2/instance-types/. Computation: and Experience, 23, Amazon EC2 EC2 instance type, http://aws.amazon.com/ec2/instance-types/. A[1] under different of users and Uk0 . It becomes when we evaluate performance ofBuyya, A and is omitted Rnumber EFERENCES online2 used in the auction design include the posted pricing framesmart grid. machine power metering and R.provisioning,” in Proc. of ACM [18] K.2011. H. Kim, A.the Beloglazov, and “Power-aware provisioning online1 of virtual machines for real-time cloud services,” Concurrency andSoCC,effectiv 2011. [2] Microsoft http://azure.microsoft.com/en-us/?rnd=1. [2] Microsoft Azure,Azure, http://azure.microsoft.com/en-us/?rnd=1. 0 2010. ofTechnical virtual machines real-time services,” Concurrency larger with the increase of Uhttp://aws.amazon.com/ec2/instance-types/. here, as the design of Afor follows the same work for truthful auctions, acloud new LP formulation and and 2012. Computation: Practice and Experience, vol. 23, no. 13, pp. online2 [19] Report, An Efficient Cloud Market Mechanism for1491–1505, Computing k . The change of the number of [19] [1] Linode, Amazon EC2https://www.linode.com/. instance type, Technical Report, Anonline Efficient Cloud Market Mechanism forprimal-dual Computing [3] Linode, [3] https://www.linode.com/. R EFERENCES Computation: and Experience, vol. 23, no. 13,gradually pp.as1491–1505, [18] K. method H. A. Beloglazov, anddeadline R. Buyya, “Power-aware provisioning [1][2]Amazon EC2 instance type, http://aws.amazon.com/ec2/instance-types/. [15] S. Zam 2011. Jobs with SoftPractice Deadlines, users doesn’t have much the ratio. As technique. Jobs with SoftKim, Deadlines, https://www.dropbox.com/s/hsit71icuen6o9j/ Microsoft Azure, solution for handling soft well Fig. 10 reflects thehttps://www.dropbox.com/s/hsit71icuen6o9j/ percentage of constraints, winners [4] L. Zhang, S. http://azure.microsoft.com/en-us/?rnd=1. Ren, C. impact Wu, andLi,on Z.“A Li, “Acompetitive truthful incentive mechanism [4] L. Zhang, S. Ren, C. Wu, and Z. truthful incentive mechanism 2011. of virtual machines for real-time cloud services,” Concurrency andvisioni [2][3]Microsoft Azure, http://azure.microsoft.com/en-us/?rnd=1. 0 main.pdf?dl=0. [19] Technical Report, An Efficient Cloud Market Mechanism for Computing main.pdf?dl=0. Linode, https://www.linode.com/. for demand response datainfluences centers,” for emergency demand in colocation data centers,” Tech.Tech. Rep., indicated inemergency Theorem 5,response a larger Uink colocation negatively theRep.,rises as approximation algorithms based on LP dual and Fenchel when the number of slots increases. The possibility of Technical Report, An Efficient Cloud Market Mechanism for Computing andConvex Experience, vol. 23, no.Cambridge 13, pp. 1491–1505,2011. [1] Amazon EC2 type,and http://aws.amazon.com/ec2/instance-types/. [20] S.Computation: Boyd andVandenberghe, L.Practice Vandenberghe, Convex optimization. Cambridge with Deadlines, https://www.dropbox.com/s/hsit71icuen6o9j/ [3] https://www.linode.com/. S. Jobs Boyd and Soft L. optimization. 2015. [4]Linode, L. Zhang, S. instance Ren, C. Wu, Z. Li, “A truthful incentive mechanism [20][19] 2015. dual. Our method for handling deadline may [16] H. Zha competitive ratio when we setZ.in cthe to“A1.truthful Wedatacan also observe becomes higher whenhttps://www.dropbox.com/s/hsit71icuen6o9j/ thesoft system spansconstraints a long period Jobs with Soft Deadlines, k 2011. university press, 2004. main.pdf?dl=0. university press, 2004. [2] Microsoft Azure, http://azure.microsoft.com/en-us/?rnd=1. [4] Zhang, S. Ren, C. Wu, Li, incentive mechanism [5] RightScale, Social Gaming Cloud: A Technical White Paper, 2013.wining for emergency demand response colocation centers,” Tech. Rep., [5] L. RightScale, Social Gaming inand thein Cloud: A Technical White Paper, 2013. 0 for trut main.pdf?dl=0. [21] Wikipedia, Convex conjugate, http://en.wikipedia.org/wiki/Convex be applicable to other auction design problems where deadline [19] Technical Report, An Efficient Cloud Market Mechanism for Computing [21] Wikipedia, Convex conjugate, http://en.wikipedia.org/wiki/Convex [20] S. Boyd and L. Vandenberghe, Convex optimization. Cambridge that the competitive ratio is still less than 5 with a large U , for emergency demand response in colocation data centers,” Tech. Rep., as there are more slots available for scheduling. In addition, a [3] Linode, https://www.linode.com/. 2015. [6] M. Hajjat, X. Sun, Y. Sung, D. Maltz, S. Rao, K. Sripanidkulchai, [6] M. Hajjat, X. Sun, Y. Sung, D. Maltz, S. Rao, K. Sripanidkulchai, andk and demand conjugate. [20] S. Boyd and L. Vandenberghe, Convex optimization. Cambridge conjugate. university press, 2004. Jobs with Soft Deadlines, https://www.dropbox.com/s/hsit71icuen6o9j/ is involved, forusers example, intodemand response auctions in a [17] A. Kan M. Tawarmalani, “Cloudward bound: for beneficial migration [4] L. Zhang, S.Social Ren, C.than Wu, the and Z. Li, “A truthful incentive mechanism [5]2015. RightScale, Gaming in thetheoretical Cloud: Aplanning Technical White Paper, 2013. small M. Tawarmalani, “Cloudward bound: planning for beneficial migration which is much better bound. number of leads higher user satisfaction. [22] Google Cluster Data, TraceVersion1, https://code.google.com/p/ [22] Google Cluster Data, TraceVersion1, https://code.google.com/p/ university press, 2004. [21] Wikipedia, Convex conjugate, http://en.wikipedia.org/wiki/Convex of enterprise applications to the Proc. of ACM 2011. [5][6] RightScale, Social Gaming Cloud: AATechnical White Paper, 2013. of enterprise applications to in thethe cloud,” Proc. of ACM SIGCOMM, 2011. for emergency demand response incloud,” colocation data centers,” Tech. Rep., smart main.pdf?dl=0. M.next Hajjat, X. Sun, Y.performance Sung, D. Maltz, S. Rao, K. Sripanidkulchai, and grid. machin We study the of in SIGCOMM, the aspects online2 googleclusterdata/wiki/TraceVersion1. googleclusterdata/wiki/TraceVersion1. Wikipedia, Convex conjugate, http://en.wikipedia.org/wiki/Convex conjugate. [20] S. Boyd and L. Vandenberghe, Convex optimization. Cambridge2010. [7] L. Zhang, Z.“Cloudward Li,Sung, and C.bound: Wu, “Dynamic resource provisioning [7] M. L. Zhang, Li, and C. Wu, “Dynamic resource provisioning inandin [21] [6] X.Z.Sun, Y. D. Maltz, S. Rao, K. Sripanidkulchai, M.Hajjat, Tawarmalani, planning for beneficial migration 2015. VII. C ONCLUSIONS of social welfare and user satisfaction. Fig. 9 shows the social [22] Google Cluster Data, TraceVersion1, https://code.google.com/p/ conjugate. university press, 2004. cloud computing: A randomized auction approach,” in Proc. of IEEE cloud computing: AGaming randomized auction approach,” inSIGCOMM, Proc. of IEEE R EFERENCES ofTawarmalani, enterprise applications toin the cloud,” Proc. of for ACM 2011. “Cloudward bound: planning beneficial migration [18] K. H. [5] M. RightScale, Social the Cloud: A Technical White Paper, 2013. googleclusterdata/wiki/TraceVersion1. Google Cluster Data, TraceVersion1, https://code.google.com/p/ INFOCOM, 2014. [21] Wikipedia, Convex conjugate, http://en.wikipedia.org/wiki/Convex INFOCOM, 2014. We studied the auction design for cloud computing jobs that welfare and cloud provider’s revenue with different number2011. ofinand [22] of virt applications to Proc. ofresource ACM SIGCOMM, [7]of L.enterprise Zhang, Z.Sun, Li, andSung, C.the Wu, “Dynamic provisioning [6] M. Hajjat, X. Y. D.cloud,” Maltz, S. Rao, K. Sripanidkulchai, googleclusterdata/wiki/TraceVersion1. 0 conjugate. Compu [1]soft Amazon EC2 instance type, http://aws.amazon.com/ec2/instance-types/. cloud computing: A auction approach,” inprovisioning Proc. ofmigration IEEE [7] L. Zhang, Li, “Cloudward andrandomized C. Wu, “Dynamic resource in have M. Tawarmalani, bound: planning for beneficial users when weZ.vary the value of U . We can obverse that both completion deadlines. Our main contribution is an k [22]Microsoft Google Azure, Cluster Data, TraceVersion1, https://code.google.com/p/2011. [2] http://azure.microsoft.com/en-us/?rnd=1. INFOCOM, 2014.A randomized cloud computing: auctionProc. approach,” inSIGCOMM, Proc. of IEEE of enterprise applications to the cloud,” of ACM 2011. the social welfare and revenue increase with the increment of online cloud job auction that is truthful and computationally [19] Technic googleclusterdata/wiki/TraceVersion1. https://www.linode.com/. [7] INFOCOM, L. Zhang, 2014. Z. Li, and C. Wu, “Dynamic resource provisioning in [3] Linode, Jobs w number of users and Uk0 . The reason for it has been explained efficient, and achieves a good competitive ratio in social [4] L. Zhang, S. Ren, C. Wu, and Z. Li, “A truthful incentive mechanism 5000W. [8] cloud computing: A randomized auction approach,” in Proc. of IEEE INFOCOM, 2014. for emergency demand response in colocation data centers,” Tech. Rep., 2015. [5] RightScale, Social Gaming in the Cloud: A Technical White Paper, 2013. [6] M. Hajjat, X. Sun, Y. Sung, D. Maltz, S. Rao, K. Sripanidkulchai, and M. Tawarmalani, “Cloudward bound: planning for beneficial migration main.p [20] S. Boy univers [21] Wikipe conjug 12 welfare. Techniques used in the auction design include the posted pricing framework for truthful online auctions, a new LP formulation and solution method for handling soft deadline constraints, as well as approximation algorithms based on LP dual and Fenchel dual. Our method for handling soft deadline constraints may be applicable to other auction design problems where deadline is involved, for example, in demand response auctions in a smart grid. R EFERENCES [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] Amazon EC2 instance type, http://aws.amazon.com/ec2/instance-types/. Microsoft Azure, http://azure.microsoft.com/en-us/?rnd=1. Linode, https://www.linode.com/. L. Zhang, S. Ren, C. Wu, and Z. Li, “A truthful incentive mechanism for emergency demand response in colocation data centers,” Tech. Rep., 2015. RightScale, Social Gaming in the Cloud: A Technical White Paper, 2013. M. Hajjat, X. Sun, Y. Sung, D. Maltz, S. Rao, K. Sripanidkulchai, and M. Tawarmalani, “Cloudward bound: planning for beneficial migration of enterprise applications to the cloud,” Proc. of ACM SIGCOMM, 2011. L. Zhang, Z. Li, and C. Wu, “Dynamic resource provisioning in cloud computing: A randomized auction approach,” in Proc. of IEEE INFOCOM, 2014. W. Shi, L. Zhang, C. Wu, Z. Li, and F. Lau, “An online auction framework for dynamic resource provisioning in cloud computing,” in Proc. of ACM SIGMETRICS, 2014. X. Zhang, Z. Huang, C. Wu, Z. Li, and F. Lau, “Online auctions in IaaS clouds: welfare and profit maximization with server costs,” in Proc. of ACM SIGMETRICS, 2015. N. Buchbinder and J. Naor, “Online primal-dual algorithms for covering and packing problems,” in ESA. Springer, 2005, pp. 689–701. Z. Huang and A. Kim, “Welfare maximization with production costs: a primal dual approach,” in Proc. of the ACM-SIAM SODA, 2015. Wikipedia, Dynamic frequency scaling, http://en.wikipedia.org/wiki/ Dynamic frequency scaling. N. R. Devanur, “Fisher markets and convex programs,” JACM, 2010. Q. Wang, K. Ren, and X. Meng, “When cloud meets ebay: Towards effective pricing for cloud computing,” in Proc. of IEEE INFOCOM, 2012. S. Zaman and D. Grosu, “Combinatorial auction-based dynamic vm provisioning and allocation in clouds,” in Proc. of IEEE Cloud CloudCom, 2011. H. Zhang, B. Li, H. Jiang, F. Liu, A. Vasilakos, and J. Liu, “A framework for truthful online auctions in cloud computing with heterogeneous user demands,” in Proc. of IEEE INFOCOM, 2013. B. Lucier, I. Menache, J. S. Naor, and J. Yaniv, “Efficient online scheduling for deadline-sensitive jobs,” in Proc. of ACM SPAA, 2013. N. Jain, I. Menache, J. S. Naor, and J. Yaniv, “Near-optimal scheduling mechanisms for deadline-sensitive jobs in large computing clusters,” ACM Transactions on Parallel Computing, vol. 2, no. 1, p. 3, 2015. Y. Azar, I. Kalp-Shaltiel, B. Lucier, I. Menache, J. S. Naor, and J. Yaniv, “Truthful online scheduling with commitments,” in Proc. of ACM EC, 2015. N. Buchbinder and J. Naor, “The design of competitive online algorithms via a primal: dual approach,” Foundations and Trends® in Theoretical Computer Science, vol. 3, no. 2-3, pp. 93–263, 2009. A. Blum, A. Gupta, Y. Mansour, and A. Sharma, “Welfare and profit maximization with production costs,” in Proc. of IEEE FOCS, 2011. A. Kansal, F. Zhao, J. Liu, N. Kothari, and A. A. Bhattacharya, “Virtual machine power metering and provisioning,” in Proc. of ACM SoCC, 2010. K. H. Kim, A. Beloglazov, and R. Buyya, “Power-aware provisioning of virtual machines for real-time cloud services,” Concurrency and Computation: Practice and Experience, vol. 23, no. 13, pp. 1491–1505, 2011. S. Boyd and L. Vandenberghe, Convex optimization. Cambridge university press, 2004. Wikipedia, Convex conjugate, http://en.wikipedia.org/wiki/Convex conjugate. Google Cluster Data, TraceVersion1, https://code.google.com/p/ googleclusterdata/wiki/TraceVersion1. optimization. energy networks. Ruiting Zhou received a B.E. degree in telecommunication engineering from Nanjing University of Post and Telecommunication, China, in 2007, a M.S. degree in telecommunications from Hong Kong University of Science and Technology, Hong Kong, in 2008 and a M.S. degree in computer science from University of Calgary, Canada, in 2012. Since March, 2016, she has been a PhD candidate at the Department of Computer Science, University of Calgary, Canada. Her research interests include smart grids, cloud computing and mobile network Zongpeng Li received his B.E. degree in Computer Science and Technology from Tsinghua University (Beijing) in 1999, his M.S. degree in Computer Science from University of Toronto in 2001, and his Ph.D. degree in ECE from University of Toronto in 2005. He has been with the University of Calgary since 2005, where he is now Professor of Computer Science. In 2011-2012, Zongpeng was a visitor at the Institute of Network Coding, Chinese University of Hong Kong. His research interests are in computer networks, network coding, cloud computing, and Chuan Wu received her B.Engr. and M.Engr. degrees in 2000 and 2002 from the Department of Computer Science and Technology, Tsinghua University, and her Ph.D. degree in 2008 from the Department of Electrical and Computer Engineering, University of Toronto, Canada. Between 2002 and 2004, She worked in the Information Technology industry in Singapore. Since September 2008, Chuan Wu has been with the Department of Computer Science at the University of Hong Kong, where she is currently an Associate Professor. Her research is in the areas of cloud computing, data center networking, online and mobile social networks, and mobile wireless networks. She was the co-recipient of the best paper awards of HotPOST 2012 and ACM e-Energy 2016. Zhiyi Huang is an assistant professor of Computer Science at the University of Hong Kong since 2014. He got his PhD under Sampath Kannan and Aaron Roth from Penn in 2013 and worked as a postdoc with Tim Roughgarden from 2013-2014. His research interest is in the area of theoretical computer science, including in particular algorithmic game theory, differential privacy, and online algorithms.
© Copyright 2026 Paperzz