Creating Very Large Scale Neighborhoods out of Smaller Ones by Compounding Moves Özlem Ergun Department of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA 30332-0205, USA James B. Orlin Operations Research Center, Massachusetts Institute of Technology, Cambridge, MA 02139, USA Abran Steele-Feldman Quantitative Ecology and Resource Management, University of Washington, Seattle, WA 981952182, USA Corresponding author: Özlem Ergun, [email protected], 404-894-2369 1 Abstract This paper discusses neighborhood search algorithms where the size of the neighborhood is “very large” with respect to the size of the input data. We concentrate on such a very large scale neighborhood (VLSN) search technique based on compounding independent moves (CIM) such as 2-opts, swaps, and insertions. We present a systematic way of creating and searching CIM neighborhoods for routing problems with side constraints. For such problems, the exact search of the CIM neighborhood becomes NP-hard. We introduce a multi-label shortest path algorithm for searching these neighborhoods heuristically. Results of a computational study on the vehicle routing problem with capacity and distance restrictions shows that CIM algorithms are very competitive approaches for solving vehicle routing problems. Overall, the solutions generated by the CIM algorithm have the best performance among the current solution methodologies in terms of percentage deviation from the best-known solutions for large-scale capacitated VRP instances. Key Words: Very-large scale neighborhood search, local search, routing 1 Introduction Given a solution S to an instance of an optimization problem, a neighborhood function defines the neighborhood, N (S) , of S. Often a neighborhood function is expressed in terms of elementary transformations of S, called “moves”, that transform S to another solution. 2-opt, swap, and insertion operations are common examples of moves. A local improvement algorithm starts with a current solution S and then searches N (S) for a better solution. If a better solution exists, then one such solution is selected to replace the current solution and the algorithm repeats. If there are no better solutions in N (S), the algorithm terminates with the current solution as the local optimum with respect to N . A critical issue in the design of a neighborhood search approach is the choice of the neighborhood structure, that is, the manner in which the neighborhood is defined. This choice largely determines whether the neighborhood search will yield solutions that are highly accurate or whether they will develop solutions with very poor local optima. In this paper, we study a class of neighborhood search algorithms where the size of the neighborhood is “very large”, often exponential, with respect to the size of the input data along with efficient algorithms for searching these neighborhoods. In particular, we develop a very large scale neighborhood, that we name the compounded independent moves (CIM) neighborhood. The research extends and generalizes earlier research independently conducted by Ergun [12] and by Potts 2 and van de Velde [24], Congram [7], and Congram et al. [8]. We concentrate on the CIM neighborhood obtained by compounding independent 2-opt, swap, and insertion moves. We show that for the travelling salesman problem, these CIM neighborhoods can be searched in polynomial time for an improving neighbor by finding a negative cost path on an auxiliary graph. Furthermore, we show that these ideas can be extended for developing CIM neighborhoods for the vehicle routing problem with side constraints. We present a computational study on the capacity and distance restricted vehicle routing problem. The compounded independent move neighborhoods applied to a variety of routing and scheduling problems are covered in more detail in [12]. For surveys of VLSN search see [2] and [9]. 2 CIM Neighborhoods for the Travelling Salesman Problem An exponentially sized neighborhood for the Travelling Salesman Problem (TSP) can be obtained by simultaneously combining a set of independent moves. Approaches that rely on combining elementary moves have been explored and surveyed for the TSP by Deineko and Woeginger [9], and applied to various sequencing problems including the TSP and some scheduling problems by Congram et al. [8], Potts and van de Velde [24], and Ergun [12]. Related TSP neighborhoods based on sequential adding and deleting of edges are studied in [13], [17], and [26]. In [29], Rego extends these ideas for the vehicle routing problem. In this section, we first explain how to construct and efficiently search the “compounded independent 2-opt neighborhood” for the symmetric TSP. Then, we extend these results for the compounded independent insertions and swaps neighborhoods. Subsequently, we enlarge these neighborhoods by weakening the independence notion for compounding moves. All of these neighborhoods have an exponential size, and we develop formulae for the growth in these neighborhoods. Our major contribution in this section is to use an auxiliary graph for searching the compounded independent 2-opt neighborhood. Potts and van de Velde [24] solve the search problem directly using dynamic programming. By our construction of the improvement graph, we can solve the search problem as a shortest path problem, and we can solve constrained search problems as constrained shortest path problems. 2.1 Compounded Independent 2-opt Neighborhood for the TSP In this subsection, we consider 2-opt moves, also called 2-exchanges. A tour T 0 is obtained from a tour T via a 2-opt move if T 0 can be obtained by adding two edges to T and deleting two edges. 3 We call T 0 a 2-opt neighbor of T . Let T be a TSP tour on n cities, and let N (T ) be the set of 2-opt neighbors of T . We present T as a sequence of cities T = i1 , i2 , ..., in , i1 . We let locT (j) denote the city in location j. In this case, locT (j) = ij . In general, T = (locT (1), ..., locT (n), locT (1)). This permits n different representations describing the same tour. If we want the description to be unique, we will specify that locT (1) = 1. We let fT (i, j) be the cost of travelling from city locT (i) to city locT (j). For the simplicity of the presentation we will drop the identifier T from the loc() and f (, ) whenever associating these functions with a specific tour is clear from the context. (Insert Figure 1) We let 2-opt-move(i, j) denote the move that transforms T into T 0 by deleting two arcs, (loc(i − 1), loc(i)), and (loc(j), loc(j + 1)), in T and adding two new arcs, (loc(i − 1), loc(j)) and (loc(i), loc(j +1)) that are not in T. This exchange also causes the order of the cities in loc (i + 1) to loc (j − 1) to be inverted. In 2-opt-move(i, j), we assume i < j. A single 2-opt move is illustrated in Figure 1, where dashed lines indicate the edges removed from the tour. (Insert Figure 2) We say that a move is restricted to the interval [i, j] if the following is true: If tour S is obtained from T by applying the move, then locS (k) = locT (k) for k < i and for k > j. 2-opt-move(i, j) and 2-opt-move(k, l) are independent if either j + 1 < k or l + 1 < i. We say that a collection S of moves are independent if every two moves in S is independent. The compounded independent 2-opt neighborhood for tour T is the set of all tours T 0 which can be obtained by applying a collection of pairwise independent 2-opt moves to T . This neighborhood is illustrated in Figure 2. Let cost(T ) be the cost of visiting the cities in the order they appear in tour T . Let c1 (i, j) denote the incremental change in tour T ’s cost after 2-opt-move(i, j) is applied. That is c1 (i, j) = cost(T 0 ) − cost(T ). The following two propositions are straight forward and stated without proof. Proposition 1 Suppose that S is a set of independent 2-opt moves. The same tour T 0 will be obtained from tour T independent of the order in which the moves in set S are carried out. Let the P cost of the moves in S be c1 (S). Then c1 (S) = cost(T 0 ) − cost(T ) = (i,j)∈S c1 (i, j). Proposition 2 Suppose that S is an ordered collection of moves such that for any two consecutive moves move(i, j) and move(k, l), j + 1 < k. Then the collection S is independent, and c1 (S) = P 1 (i,j)∈S c (i, j). 4 The compounded independent 2-opt neighborhood can be searched by finding a shortest path on a related network structure that we call the improvement graph as per [32]. Given a TSP tour T =(loc (1) , loc (2) , ..., loc (n) , loc (1)), the improvement graph G (V, E) associated with T is defined by the node set V = {1, 2, ..., n, n + 1} where each node i represents the city loc (i) for 1 ≤ i ≤ n, and node n + 1 represents the fact that the loop is completed by returning back to city loc (1). An arc (i, j) in the improvement graph represents 2-opt-move(i + 1, j − 1), a move restricted to the interval [i + 1, j − 1], that is involving only cities loc (i + 1), loc (i + 2) , ..., loc (j − 1). The arc set E consists of all arcs (i, j) such that 1 ≤ i ≤ n and j = i + 1 or i + 3 ≤ j. Note that the arc set E only consists of forward arcs, which ensures that G is acyclic and which in turn ensures by Proposition 2 that the compounded moves that are identified with the search algorithm are independent. Furthermore, we will associate an independent set of 2-opt moves with a path from 1 to n+1. There are no arcs of the form (i, i + 2) since there is no 2-opt move involving only city loc (i + 1). For convenience, we let f (n, n + 1) = f (n, 1) . The cost structure on the arcs in E is constructed as follows: • c1 (i, i + 1) = 0. Arc (i, i + 1) for 1 ≤ i ≤ n will be in the path when cities loc (i) and loc (i + 1) are not involved in any moves. • Arc (i, j) for 1 ≤ i < n − 1 and i + 2 < j ≤ n + 1 represents a 2-opt-move(i + 1, j − 1). The cost on these arcs is: c1 (i, j) = −(f (i, i + 1) + f (j − 1, j)) + (f (i, j − 1) + f (i + 1, j)). Note that when we are concerned with the asymmetric TSP the cost term should also include the impact of the reversal of the subtour (loc (i + 1) , ..., loc (j − 1)). Figure 3 illustrates a partial improvement graph for compounded 2-opts. Theorem 3 There is a one-to-one cost-preserving correspondence between paths from node 1 to node n+1 in the compounded independent 2-opt improvement graph G (V, E) and sets of independent 2-opt moves. Proof We first suppose P is a path from 1 to n + 1 in G. Let P 0 be the set of edges obtained by deleting all edges of the form (i, i+1) in P . Then the edges of P 0 correspond to an ordered collection of 2-opt moves such that for any two consecutive moves move(i, j) and move(k, l), j + 1 < k. By 5 Property 2, this collection of moves is independent, and its cost is equal to P P 1 1 (i+1,j−1)∈S c (i + 1, j − 1) = (i,j)∈S c (i, j) which is also the cost of path P . We next suppose we are given a set S of ordered independent 2-opt moves. Let P 0 be the corresponding edges in E, ordered in increasing order of index. We extend P 0 to a path P by P 1 adding edges of the form (i, i + 1). Then the cost of P is equal to c(S) = (i,j)∈S c (i, j) = P 1 • (i+1,j−1)∈S c (i + 1, j − 1). Finding any negative cost path from node 1 to node n + 1 on the improvement graph G (V, E) corresponds to finding a combination of independent 2-opt moves that transforms the current tour T into a new tour T 0 ∈ N (T ) with a lower cost. If the length of the shortest path from 1 to n + 1 has a cost of 0, then there are no improving neighbors of T , and this is a locally optimal tour under the compounded independent 2-opt neighborhood. Searching the compounded independent ¡ ¢ 2-opt neighborhood takes O n2 time if the arc costs on E either are given or can be calculated in constant time per arc. (Insert Figure 3) Although the compounded independent move neighborhood is exponentially large, it shares a key feature with the much smaller 2-opt neighborhood. Theorem 4 The set of local optimal solutions with respect to the compounded independent 2-opt neighborhood is the same as the set of local optimal solutions for the 2-opt neighborhood. Proof See [24], [12]. 2.2 • Compounded Independent Insertion and Swap Neighborhoods for the TSP The compounded independent 2-opt neighborhood can be directly extended to the move types swaps and insertions as well as others. The swap neighborhood of a given tour T = (locT (1), ...., locT (n), locT (1)) consists of all tours T 0 which can be obtained by switching the locations of two cities locT (i) and locT (j) in T . Thus locT 0 (i) = locT (j), locT 0 (j) = locT (i), and locT 0 (k) = locT (k) for all k 6= i or j. Equivalently, links (loc (i − 1) , loc (i)), (loc (i) , loc (i + 1)), (loc (j − 1) , loc (j)), and (loc (j) , loc (j + 1)) in T are broken and the links (loc (i − 1) , loc (j)), (loc (j) , loc (i + 1)), (loc (j − 1) , loc (i)) and (loc (i) , loc (j + 1)) are established. We refer to the swap as swap- move(i, j), and we assume i < j. We say that swap-move(i, j) is independent of swap-move(k, l) 6 if j + 1 < k or l + 1 < i. The compounded independent swaps neighborhood for tour T is the set of all tours T 0 which can be obtained by applying a collection of independent swap moves to T . Propositions 1 and 2 of Section 2.1 directly extend for a collection of independent swap moves. Moreover, the improvement graph for swaps is identical to the one described in Section 2.1 with the exception of the arc costs. The arc costs c2 (i, j) in this improvement graph are calculated as follows: • c2 (i, i + 1) = 0. Arc (i, i + 1) for 1 ≤ i ≤ n will be in the path when cities loc (i) and loc (i + 1) are not involved in any moves. • Arc (i, j) for 1 ≤ i < n − 1 and i + 2 < j ≤ n + 1 represents a swap-move(i + 1, j − 1). The cost on these arcs is: c2 (i, j) = −(f (i, i + 1) + f (i + 1, i + 2) + f (j − 2, j − 1) + f (j − 1, j)) + (f (i, j − 1) + f (j − 1, i + 2) + f (j − 2, i + 1) + f (i + 1, j)). The insertion neighborhood of a given tour T = (locT (1), ...., locT (n), loc(1)) consists of all tours T 0 which can be obtained by ejecting a city from its current location and inserting it into a different location. Suppose i < j. We let insertion-move(i, j) be the move such that the node locT (i) is inserted immediately after the node locT (j). The resulting tour T 0 has locT 0 (k) = locT (k) for all k < i and k > j. Also, locT 0 (k) = locT (k +1) for k = i to j −1, and locT 0 (j) = locT (i). Equivalently, the links (loc (i − 1) , loc (i)), (loc (i) , loc (i + 1)), (loc (j) , loc (j + 1)) in T are broken and the links (loc (i − 1) , loc (i + 1)), (loc (j) , loc (i)) and (loc (i) , loc (j + 1)) are established. Independence of insertion moves is defined the same as for 2-opt moves and swap moves. The compounded independent insertions neighborhoods we use in this paper is the set of all tours T 0 that can be obtained by applying a collection of independent forward insertion moves where a city loc(i) is ejected from its current location and is inserted between two cities loc(j) and loc(j + 1) for all i < j −1. Propositions 1 and 2 of Section 2.1 directly extend for a collection of independent insertion moves. Moreover, the improvement graph for insertion moves is identical to the one described in Section 2.1 with the exception of the arc costs. The arc costs c3 (i, j) in this improvement graph are calculated as follows: • c3 (i, i + 1) = 0. Arc (i, i + 1) for 1 ≤ i ≤ n will be in the path when cities loc (i) and loc (i + 1) are not involved in any moves. 7 • Arc (i, j) for 1 ≤ i < n − 1 and i + 2 < j ≤ n + 1 represents an insertion-move(i + 1, j − 1). The cost on these arcs is: c3 (i, j) = −(f (i, i + 1) + f (i + 1, i + 2) + f (j − 1, j)) + (f (i, i + 2) + f (j − 1, i + 1) + f (i + 1, j)). For i < j, we could also consider a reverse-insertion move in which the node locT (j) is inserted directly before the node locT (i). We can compute the cost c4 (i, j) of reverse-insertionmove(i + 1, j − 1) in a similar manner to computing the cost c3 (i, j). Given the above structure of the improvement graphs for the compounded independent swap and insertion neighborhoods, it is easy to see that the Theorems 3 and 4 can directly be extended for these neighborhoods as well. Furthermore, searching for an improving neighbor can be done in O(n2 ) time by finding a shortest path on the appropriate improvement graph. Moreover, one can accommodate the compounding of different types of moves in one neighborhood. For example, a larger neighborhood obtained by compounding independent 2-opt, swap, insertion, and reverse-insertion moves can be modelled and searched by constructing an identical improvement graph and letting the arc cost c(i, j) = min(c1 (i, j), c2 (i, j), c3 (i, j), c4 (i, j)). 2.3 Compounded Weakly Independent Moves Neighborhoods By weakening the independence requirement for compounding moves one can obtain larger sized neighborhoods. We say that 2-opt-move(i, j) is weakly independent of 2-opt-move(k, l) if j + 1 ≤ k or l + 1 ≤ i. We define weak independence for swap, insertion, and reverse-insertion moves in the same manner. Then the compounded weakly independent 2-opt (swap, insertion, reverse-insertion) neighborhood for tour T is the set of all tours T 0 which can be obtained by applying a collection of weakly independent 2-opt (swap, insertion, reverse-insertion) moves to T . In general, the size of the compounded weakly independent moves neighborhood is asymptotically larger than the size of the compounded independent moves neighborhood. We give asymptotic bounds for the exponential growth of these neighborhoods in Section 2.4. At the same time, the increased size of these neighborhoods comes at a cost in search time. The compounded weakly independent 2-opt moves neighborhood apparently takes n times as long to search as the compounded independent 2-opt moves neighborhood. (At least, we could not find a faster approach.) The same increase in running time applies for the swap-based neighborhoods. However, we will 8 show below that the compounded weakly independent insertion neighborhood takes only a constant times longer to search as the compounded independent insertion neighborhood. Let G = (V, E) be a graph with arc costs c(ej ) for ej ∈ E. A turn penalty (see [20]) is a function g : E × E → <. If e ∈ E and e0 ∈ E, then g(e, e0 ) is the (additional) cost of having arc e0 directly following arc e in a path. Then the cost of a path consisting of arcs e1 , ..., ek is Pk Pk−1 j=1 c(ej ) + j=1 g(ej , ej+1 ). The compounded weakly independent 2-opt, swap, insertion, and reverse-insertion neighborhoods can be searched by finding shortest paths with turn penalties on improvement graphs G(V, E) constructed similarly to the ones described in the previous sections. As before, let T be the current tour. In these improvement graphs, the node set is V = {1, 2, ..., n, n + 1}, where each node i represents the city locT (i) for 1 ≤ i ≤ n, and node n + 1 represents the fact that the loop is completed by returning back to city locT (1). An arc (i, j) in the improvement graph represents 2-opt-move(i, j − 1), which is a move that involves nodes locT (i), locT (i + 1) , ..., locT (j − 1). The arc set E consists of all arcs (i, j) such that 1 < i ≤ n and i + 1 ≤ j. The cost on the improvement graph arcs depend not only on the move they represent but also on the move represented by the previous arc. The turn penalties for the arc (j + 1, l) preceded by an arc (i, j + 1) in the CWI 2-opt, swap, insertion, and reverse-insertion neighborhood improvement graphs can be calculated as follows: • For arc (j + 1, j + 2) the turn penalties for the 2-opt, swap, insertion, and reverse-insertion moves are equal to 0 since these arcs imply that node j + 1 is not part of a move. • For arcs (j + 1, l) where l > j + 2, for the CWI 2-opt neighborhood: f (j, j + 1) + f (i, l − 1) − f (i, j + 1) − f (j, l − 1) for the CWI swap neighborhood: f (j, j + 1) + f (i, l − 1) − f (i, j + 1) − f (j, l − 1) for the CWI insertion neighborhood: f (j, j + 1) + f (i, j + 2) − f (i, j + 1) − f (j, j + 2) for the CWI reverse-insertion neighborhood: f (j, j + 1) + f (j − 1, l − 1) − f (j − 1, j + 1) − f (j, l − 1). 9 The following theorem is a direct extension of Theorem 1. Theorem 5 There is a one-to-one correspondence between paths from node 1 to node n+1 in the compounded weakly independent 2-opt (swap, insertion, reverse-insertion) improvement graph and sets of weakly independent 2-opt (swap, insertion, reverse-insertion) moves. Moreover, the cost of the weakly independent 2-opt (swap, insertion, reverse-insertion) moves is the same as the cost of the path taking into account turn penalties. In general, these shortest paths with turn penalties can be found in O(n3 ) time (for example see Exercise 4.40 on page 130 of [3]). However the following theorem shows that the quality of the local optimal solutions obtained by CWI moves is at least as good as the ones obtained by CI moves. Theorem 6 For n ≥ 5, a locally optimal solution for the compounded weakly independent 2opt (swap, insertion, reverse-insertion) neighborhood is also locally optimal for the 2-opt (swap, insertion, reverse-insertion) neighborhood. However, there exists locally optimal solutions for the 2opt (swap, insertion, reverse-insertion) neighborhood that are not locally optimal for the compounded weakly independent 2-opt (swap, insertion, reverse-insertion) neighborhood. Proof The first claim is trivially true, since every single 2-opt (swap, insertion, reverse-insertion) move is also a feasible move for the CWI 2-opt (swap, insertion, reverse-insertion) neighborhood. We will prove the second claim by giving a simple example where a given TSP tour is locally optimal with respect to the 2-opt neighborhood but not with respect to the CWI 2-opt neighborhood. Consider the tour T = (1, 2, 3, 4, 5, 6, 7, 8, 1) on eight cities. Let the cost on arc (i, j) be 2 for all i and j except for the city pairs (2, 7), (1, 6), and (3, 8). Assume that the cost on arc (2, 7) is 1 and the costs on arcs (1, 6) and (3, 8) are 4. We claim that tour T is a locally optimal tour under the 2-opt neighborhood. Any 2-opt move deletes two edges in T with a total cost of 4. For the 2-opt move to be profitable the total cost of the two edges added must be less than 4. This can only be achieved if edge (2, 7) is one of the two edges added to the tour. However, if edge (2, 7) is one of the edges added then either edge (1, 6) or edge (3, 8) must also be added. The total cost of the edges added in both cases is equal to 5 which makes any 2-opt move involving the edge (2, 7) non-profitable. Hence there does not exist a profitable 2-opt move for tour T . On the other hand, consider 2-opt-move(2, 4) and 2-opt-move(5, 7). Applying these two moves on T simultaneously, we obtain tour T 0 = (1, 4, 3, 2, 7, 6, 5, 8, 1). By definition T 0 is in the CWI 10 2-opt neighborhood of T . Furthermore, since the cost of T is equal to 16 and the cost of T 0 is equal to 15, T is not a local optima under the CWI 2-opt moves. Examples can be constructed similarly where a locally optimal tour under swap, insertion, and reverse-insertion neighborhoods is not locally optimal under the CWI swap, insertion, and reverse-insertion neighborhoods, respectively. • The turn penalties for the CWI insertion neighborhood have a simpler form, and this neighborhood can be searched in O(n2 ) on a specially constructed improvement graph as follows. Let G(V ∗ , E ∗ ) be an improvement graph for the compounded weakly insertions neighborhood. Let V ∗ = V ∪ V 0 , where V = {1, 2, ..., n, n + 1} represents the n nodes of the current tour T and V 0 = {20 , 30 , ..., n − 20 , n − 10 } is a copy of the nodes 2 through n − 1. As previously done, we will construct G∗ so that there is a one-to-one correspondence between the paths in G∗ from 1 to n + 1 and weakly independent sets of insertion moves. We describe the correspondence as we describe the construction of E ∗ . We let P denote a path from 1 to n + 1 in G∗ , and we let T denote the current tour. • (i, i + 1) ∈ E ∗ with cost 0 for 1 ≤ i ≤ n. If (i, i + 1) ∈ P , this is interpreted as locT (i) and locT (i + 1) are not in an insertion move. • (i, j) ∈ E ∗ for 1 ≤ i < n − 1 and i + 2 < j ≤ n + 1. If (i, j) ∈ P , this is interpreted as insertion-move(i + 1, j − 1). The cost on arc (i, j) is c5 (i, j) = −(f (i, i + 1) + f (i + 1, i + 2) + f (j − 1, j)) + (f (i, i + 2) + f (j − 1, i + 1) + f (i + 1, j)). • (i, j 0 ) ∈ E ∗ for 1 ≤ i < n − 3 and i + 2 < j ≤ n − 1. If (i, j 0 ) ∈ P , this is interpreted as insertion-move(i + 1, j) followed directly by insertion-move(j, k) for some k ≥ j + 1. Thus locT (i + 1) will be inserted between locT (j − 1) and locT (j + 1). The cost on arc (i, j 0 ) is c5 (i, j 0 ) = −(f (i, i + 1) + f (i + 1, i + 2) + f (j − 1, j) + f (j, j + 1)) + (f (i, i + 2) + f (j − 1, i + 1) + f (i + 1, j + 1)). • (i0 , j 0 ) ∈ E ∗ for 2 ≤ i < n−3 and i+2 < j ≤ n−1. If (i0 , j 0 ) ∈ P , this is interpreted as inserting locT (i) between locT (j − 1) and locT (j + 1) simultaneously with an insertion-move(j, k) for some k ≥ j + 1. The cost on arc (i0 , j 0 ) is c5 (i0 , j 0 ) = −(f (j − 1, j) + f (j, j + 1)) + (f (j − 1, i) + f (i, j + 1)). 11 • (i0 , j) ∈ E ∗ for 2 ≤ i < n − 1 and i + 2 < j ≤ n + 1. If (i0 , j) ∈ P , this is interpreted as inserting locT (i) between locT (j − 1) and locT (j). The cost on arc (i0 , j) is c5 (i0 , j) = −(f (j − 1, j)) + (f (j − 1, i) + f (i, j)). The neighborhood is searched by finding a shortest path in G(V ∗ , E ∗ ) from node 1 to node n + 1. Figure 4 illustrates an example with a partial improvement graph for the compounded weakly independent insertions. (Insert Figure 4) 2.4 Size of the Neighborhood The number of neighbors in the compounded independent 2-opt (swap, insertion, reverse-insertion) neighborhood is equal to the number of paths from node 1 to n + 1 in the improvement graph described in Section 2. We give a recursion for this number next, and solve the recursion to provide asymptotic formulae for the number of paths. Let g (n) be the number of paths from node 1 to node n + 1 and g (n − 1) be the number of paths from node 1 to node n in the improvement graph. Then g (n) = g (n − 1) + g (n − 3) + g (n − 4) + ... + g (1) + g(0). (1) This can be shown by observing that the first arc of the path goes from node 1 to one of the following nodes: 2, 4, 5, ..., n + 1. (Note that there are no arcs between node 1 and node 3 since the improvement graph does not have arcs of the form (i, i + 2) .) Then the number of paths from node i ∈ {2, 4, 5, ..., n + 1} to node n + 1 is g (n − i + 1). From the above recursion it follows that: g (n − 1) = g (n − 2) + g (n − 4) + g (n − 5) + ... + g (1) + g(0). (2) Subtracting (2) from (1), we get an equivalent third order linear recurrence g (n) = 2g (n − 1) − g (n − 2) + g (n − 3) ⇒ g(n) − 2g (n − 1) + g (n − 2) − g (n − 3) = 0 with the boundary conditions g (0) = g (1) = 1. 12 (3) (4) Using standard results in solving linear recurrence relations (see, e.g. pages 312-314 of [22]) we obtain the characteristic equation of recurrence (4), which is equal to x3 − 2x2 + x1 − 1 = 0. (5) The largest real root of the characteristic equation (5) is between 1.7548 and 1.7549. Again by standard results in solving linear recurrences, we conclude that the largest homogeneous solution for the recurrence (4) is bounded below by A × 1.7548n and bounded above by A × 1.7549n for some constant A and for all sufficiently large values of n. Hence the size of the compounded independent moves neighborhood is O (1.7549n ) and Ω (1.7548n ). A similar result is also presented in [7]. Furthermore, if the union of k different neighborhoods is taken as described in Section 2.2 then the size of this larger neighborhood can be calculated by solving the following recursion: g (n) = 2g (n − 1) − g (n − 2) + kg (n − 3) ⇒ g(n) − 2g (n − 1) + g (n − 2) − kg (n − 3) = 0 with the boundary conditions g (0) = g (1) = 1. In the case of k = 2, the solution to this recurrence relation is 2n . For the CWI 2-opt (swap, insertion, reverse-insertion) neighborhood, the recursion is as follows: g(n) = g(n − 1) + g(n − 2) + ... + g(1). This reduces to g(n) = 2g(n − 1), with a root of 2. Thus the number of neighbors is θ(2n ), which is asymptotically greater than the number of neighbors for the CI 2-opt (swap, insertion, reverse-insertion) neighborhood. 3 The Vehicle Routing Problem In this section, we consider the vehicle routing problem (VRP) and generalize the CIM neighborhood of the previous section. We first describe the notation and constraints for the VRP. We consider a vehicle routing problem with m vehicles in which cities 1, 2, ..., n need to be visited. The distance from city i to city j is d(i, j). The maximum distance that vehicle k is permitted to travel is Dk for k = 1 to m. There is a demand qi and an additional distance penalty bi at each city i, and a capacity Ck for vehicle k for each k = 1 to m. There is a single depot that is labelled as city 0. A routing T = (T1 , T2 , ..., Tm ) is a collection of m subtours such that each city in {1, 2, ..., n} appears in exactly one of the subtours, and each subtour begins and ends with the 13 depot. The VRP with distance constraints is formulated as follows: Pm P min k=1 (i,j)∈Tk P subject to P (i,j)∈Tk i∈Tk qi d(i, j) + d(i, j) ≤ Ck for k = 1 to m P i∈Tk bi ≤ Dk for k = 1 to m (T1 , ..., Tm ) is a routing. Any routing T satisfying the first and second constraints is called a feasible routing. The VRP models the problem of determining optimal delivery or collection routes of a truck fleet through a set of locations. The VRP is NP-hard, and is very difficult to solve in practice. Problems with more than 50 cities are rarely solvable to optimality [27]. As a result of the VRP’s inherent intractability, there is a large body of research on heuristics, as surveyed in chapters 9 and 10 of [1]. (See [14], [16], [21], and [35] for surveys on the VRP.) Our generalization of the CIM neighborhood to the VRP has restrictions, and relies on heuristics. There are two fundamental difficulties that arise in any attempt to generalize the CIM to the VRP. 1. Proposition 1 of Section 2.1 fails to hold because of capacity and distance constraints. In general, applying a collection of independent moves to a VRP solution may lead to an infeasible solution. 2. Proposition 2 of Section 2.1 relies on there being a complete order on cities. This property can be generalized, in multiple ways, and it is not obvious how to generalize it in the best way. We address these difficulties as follows: 1. We will generalize the notion of “independence” of moves, and design a heuristic shortest path algorithm that simultaneously selects an independent set of moves and guarantees that capacity and distance constraints are satisfied. 2. We permit any partial order ≺ of the cities satisfying the following relationship: if cities i and j are both in the same subtour, and if city i precedes city j, then i ≺ j. This creates a partial order on the n cities. We then extend this partial order to a complete order, which is also denoted by ≺. In our improvement graph, if there is an arc (i, j), then i ≺ j. 14 Thompson and Psaraftis [33], developed an alternative approach for applying very large scale neighborhood search to the VRP based on the cyclic exchange neighborhood (see also, Thompson and Orlin [32], and Ahuja et al [4]). This technique also relies on a search of an appropriately defined improvement graph. The ideas we develop here on heuristic search of the CIM neighborhoods for the capacitated VRP can easily be extended to routing and scheduling problems with various side constraints (e.g. time windows, precedence constraints, etc.). For a detailed study of these extensions see [12]. 3.1 The Improvement Graph and the Search Algorithm In this subsection, we introduce our notation for describing the routing T , its neighborhood, and the improvement graph. Let T = (T1 , ..., Tm ) be a feasible routing. We let the set N = {1, 2, ..., n} denote the set of non-depot cities. We will refer to elements of N as cities, except when they appear in the improvement graph, in which case they will be called nodes. For each city i ∈ N , predCity(i) denotes the city that immediately precedes i on the subtour containing city i, and sucCity(i) denotes the city that immediately follows i on its subtour. We say that the cities in N are order-consistent with T if predCity(i) < i for each i ∈ N . This implies that i < sucCity(i) with the exception of when sucCity(i) = 0. We will construct an improvement graph with respect to a routing T. Before describing the improvement graph, we make a simplifying assumption. Order Assumption: We assume that the cities in N are order-consistent with T. We note that if the cities in N are not order consistent with T, then there are exponentially many ways of relabelling the nodes of N so that they become order consistent. Of these ways, there are two relabelling that we give special attention to. We say that the cities in N are single tour ordered if they are order consistent with T , and if r < s, then each city in Tr has an index less than each city in Ts . (We call this labelling “single tour ordered” because we can view the sequence 1, 2, ..., n as a tour, and if we ignore the depot, then each subtour occurs as consecutive cities on this tour.) We say that the cities in N are position ordered if they are order consistent and if for i < j, the city in position i of subtour Tr has a label that is less than the city in position j of subtour Ts , for all r and s. We will create three different VRP improvement graphs. The first graph will be referred to as the ST-improvement graph. For the ST-improvement graph, we assume that the nodes of N have 15 been relabeled so that they are single tour ordered with respect to T . We let G = (V, E) denote the ST-improvement graph. The node set is V = N ∪ {0k : k = 1 to m + 1} ∪ {s}. There is a node in V for each node in N and for the first node (the depot) of each subtour. In addition, there is a node s for the origin node of the path and a final node 0m+1 . For each node j 6= s and j 6= 0m+1 , we create two arcs (s, j) and (j, 0m+1 ) with 0 cost. These arcs correspond to the null move. We now define pred( ) and suc( ) for G as follows. If i is not the first non-depot node of its subtour, then pred(i) = predCity(i). If i is the first non-depot node of subtour Tk , then pred(i) = 0k . Similarly, if i is not the last non-depot node of its subtour, then suc(i) = sucCity(i). If i is followed by the depot on Tk and k < m, then suc(i) = 0k+1 . There is an arc (pred(i), i) for each node i in G, and the cost of this arc is 0. There are no arcs which imply moves involving the depot nodes. For each pair i, j ∈ N with i < j, swap-move(i, j) represents the swapping of cities i and j. Thus i is deleted from its subtour, and j is inserted in its place. And j is deleted from its subtour and i is inserted in its place. For each swap-move(i, j), we create an arc (pred(i), suc(j)) in the ST-improvement graph, and its cost c1 (pred(i), suc(j)) is the decrease in the objective function obtained by making the move. More precisely: c1 (pred(i), suc(j)) = −(d(pred(i), i) + d(i, suc(i)) + d(pred(j), j) + d(j, suc(j))) + (d(pred(i), j) + d(j, suc(i)) + d(pred(j), i) + d(i, suc(j))). For each i, j ∈ N with i < j, insertion-move(i, j) represents the deletion of node i from its subtour and its insertion between cities j and suc(j) in their subtour. For each insertion-move(i, j), we create an arc (pred(i), suc(j)) in the ST-improvement graph, and its cost c2 (pred(i), suc(j)) is the decrease in the objective function by making the move. More precisely: c2 (pred(i), suc(j)) = −(d(pred(i), i) + d(i, suc(i)) + d(j, suc(j))) + (d(pred(i), suc(i)) + d(j, i) + d(i, suc(j))). For each i, j ∈ N with i < j, reverse-insertion-move(i, j) represents the deletion of node j from its subtour and its insertion between cities i and pred(i) in their subtour. For each reverseinsertion-move(i, j), we create an arc (pred(i), suc(j)) in the ST-improvement graph, and its cost 16 c3 (pred(i), suc(j)) is the decrease in the objective function by making the move. More precisely: c3 (pred(i), suc(j)) = −(d(pred(i), i) + d(pred(j), j) + d(j, suc(j))) + (d(pred(i), j) + d(j, i) + d(pred(j), suc(j))). We let c(i, j) = min(c1 (i, j), c2 (i, j), c3 (i, j)) for each i, j ∈ V , with i < j. The ST-improvement graph has O(n) nodes, O(n2 ) arcs, and can be determined in O(n2 ) time. A partial ST-improvement graph is presented in Figure 5(a). We say that swap-move(i, j) (respectively, insertion-move(i, j) and reverse-insertion-move(i, j)) is independent of swap-move(k, l) (respectively, insertion-move(k, l) and reverse-insertion-move(i, j)) if suc(j) < k or if suc(l) < i. Theorem 7 Suppose that T is a routing, and that the nodes are order consistent with respect to T . Let G = (V, E) be an ST-improvement graph for the VRP. If P is a path in G from s to 0m+1 , then P corresponds to an independent set S of moves. Moreover, let c(P ) be the cost of path P and d(T ) be the total distance travelled in routing T . If the (possibly infeasible) routing T 0 is obtained from T by applying the moves in S, then d(T 0 ) − d(T ) = c(P ). Proof Suppose that P is a path from s to 0m+1 in G. Let P 0 be the set of edges obtained by deleting all edges of the form (pred(i), i) in P . Then the edges of P 0 correspond to an ordered collection of moves such that for any two consecutive moves move(i, j) and move(k, l), suc(j) < k. Hence P this collection of moves is independent, and its cost is equal to the cost of path P , (i,j)∈S c(i, j), which is also equal to d(T 0 ) − d(T ). • Theorem 7 states that any path in G from s to 0m+1 induces a set of compounded independent moves leading to a new routing T 0 . However, T 0 is possibly infeasible because it may violate the capacity constraints and it may violate the distance constraints for some subtour. Our way of obtaining feasible routings is to ensure that for each path P determined by the algorithm, we obtain a feasible tour T 0 if we apply the compounded independent moves implied by P to T . We discuss how to maintain feasibility in Subsection 3.3. As for the TSP (see Section 2.3), by weakening the independence requirement for compounding moves, one can obtain larger sized neighborhoods for the VRP. We say that swap (respectively, insertion and reverse-insertion) move(i, j) is weakly independent of swap (respectively, insertion and reverse-insertion) move(k, l) if suc(j) ≤ k or suc(l) ≤ i. The compounded weakly independent swap 17 (respectively, insertion and reverse-insertion) neighborhoods can be searched by finding shortest paths with turn penalties on improvement graphs constructed similarly to the ones described above. The turn penalties are calculated in the same manner as in Section 2.3. 3.2 Other Improvement Graphs The PO-improvement graph is defined in exactly the same manner as the ST-improvement graph, except that at the beginning we relabel the nodes so that they are position ordered. Because the nodes have different labels, the improvement graph has a different set of arcs. All arcs (i, j) such that i < j are present in the PO-improvement graph with a few exceptions. Recall that in the TSP improvement graphs no arcs of the form (i, i + 2) were included since these arcs do not correspond to a well defined move. Similarly in the PO-improvement graph, arcs of the form (i, j) are not allowed when i = pred(k) and j = suc(k) for some k. Also, to maintain an acyclic graph, we do not allow arc (i, j) if i and j have the same position in their respective subtours. All arcs (i, j) such that i and predCity(j) have the same position in their respective subtours imply null moves and have a cost of 0. A partial PO-improvement graph is presented in Figure 5(b). We also considered an enhancement to the swap neighborhood that is accomplished by exchanging consecutive nodes of cities. The improvement graph for the multi-city swap (MCS) neighborhood (we refer to this improvement graph as the MCS-improvement graph) is constructed as the ST-improvement graph with the addition of nodes representing a set of consecutive cities. For example, the node < i, j > represents the consecutive set of cities starting with node i in its subtour and ending with city j. (This presumes that i and j are in the same subtour, and that i < j.) A swap-move(< i, j >, < k, l >) represents the deletion of the cities represented by < i, j > from its subtour and inserting the cities represented by < k, l > in between cities pred(i) and suc(j). At the same time, there is a deletion of the cities represented by < k, l > from its subtour and inserting the cities represented by < i, j > in between cities pred(k) and suc(l). In terms of the improvement graph, we let pred(< i, j >) = pred(i), and we let suc(< i, j >) = suc(j). The arcs and costs for this improvement graph are defined in a similar manner to ST-improvement graphs. We note that Theorem 7 and the implementation of compounded weakly independent moves directly extend to the PO-improvement graph and the MCS-improvement graph. (Insert Figure 5 here.) 18 3.3 Partial Demands and Partial Capacities of Compounded Moves Let G = (V, E) be an improvement graph based on a routing T , and let P be any path in G P } be the routing obtained by applying the moves with initial node 01 . Let T P = {T1P , T2P , ..., Tm P induced by P , as described in Theorem 7. We let q(P, k) = i∈T P qi . Thus, q(P, k) is the amount k of capacity used up on the k-th subtour after making the moves induced by P . Similarly, let P d(P, k) = i∈T P (d(i, suc(i)) + bi ). Thus d(P, k) is the distance accrued in the k-th subtour after k Pm P making the moves induced by P . We let d(P ) = k=1 i∈T P d(i, suc(i)). Thus d(P ) is the k objective function value after making the moves induced by P. We say that path P is capacity feasible if q(P, k) ≤ Ck for k = 1 to m. We say that path P is distance feasible if d(P, k) ≤ Dk for each k. Suppose that P = P 0 ∪ (i, j), that is, P is obtained by appending arc (i, j) to path P 0 . Suppose that arc (i, j) represents a move that involves Tk and Tl . Then d(P, r) = d(P 0 , r) for r 6= k or l, and q(P, r) = q(P 0 , r) for r 6= k or l. Moreover, d(P, k) can be computed from d(P 0 , k) in O(1) time through a straightforward incremental calculation of the effect of the last move in P . Similarly, d(P, l), q(P, k), and q(P, l) can be computed in O(1) time incrementally from d(P 0 , l), q(P 0 , k), and q(P 0 , l). 3.3.1 The Heuristic Shortest Path Algorithm The following algorithm is a variant of the dynamic programming algorithm for solving the shortest path problem on an acyclic graph. We heuristically ensure that the capacity and distance constraints are satisfied. We let f (j) denote the distance label associated with node j for each j ∈ V . We let Pj denote the path from s to j determined by the algorithm. We let E(i) denote the arcs in E that emanate from the node i. We let parent(i) be the node that precedes j on the path Pj . 19 Procedure Shortest Path begin Ps := s; f (s) := 0; for each r = 1 to m, do d(Ps , r) := d(Tr ) and q(Ps , r) := q(Tr ); for each j ∈ V such that j 6= s, do f (j) := c(s, j) and parent(j) := s; for each j ∈ V \ s do begin i := parent(j); Pj := Pi ∪ (i, j); determine d(Pj , r) for all r = 1 to m; determine q(Pj , r) for all r = 1 to m; for each arc (j, k) ∈ E(j) do if f (k) > f (j) + c(j, k) and if the path Pj ∪ (j, k) is both distance and capacity feasible then f (k) := f (j) + c(j, k) and parent(k) := j; end end Procedure Shortest Path is a multiple label keeping heuristic based on a label setting shortest path algorithm (see Chapter 4 in [3] ) for partially searching ST, PO, and MCS improvement graphs. Such multiple label keeping heuristics are widely used in the literature for solving constrained shortest paths problems (see, for example, [10]). Theorem 8 Procedure Shortest Path determines a path Pj from s to j for all j such that Pj is both distance and capacity feasible. The time to run the shortest path procedure is O(n2 ). Proof There is a trivial path consisting of the arc (s, j). Moreover, any other path Pj from s to j obtained by the algorithm is guaranteed to be distance and capacity feasible. We now consider the computation time. Immediately before the arcs in E(j) are scanned in the for loop, we need to determine d(Pj , r) for all r, and we need to determine q(Pj , r) for all r. In the algorithm, Pj = Pi ∪ (i, j). The values d(Pi , r) and q(Pi , r) have already been calculated for all r prior to this step. So, d(Pj , r) and q(Pj , r) can be calculated incrementally in O(1) step, and all values can be calculated in O(nm) 20 steps. Since the number m of vehicles is less than the number n of cities, this is dominated by O(n2 ). Each arc (j, k) is scanned once in the inner for loop. When an arc is scanned, we need to determine whether the path Pk = Pj ∪ (j, k) is distance and capacity feasible. By the above, we can determine feasibility incrementally from d(Pj , ) and q(Pj , ) in O(1) steps. Thus scanning each arc and determining its feasibility can be done in O(1) steps. Since there are O(n2 ) arcs, the running time is O(n2 ) steps in total, completing the proof. • The Procedure Shortest Path always finds a feasible path on the ST, PO, and MCS improvement graphs with a non-positive cost. However, to be able to solve problems with a fixed fleet size, in the computational experiments below, we use a modified version of the Procedure Shortest Path where infeasible solutions are allowed but penalized. Furthermore, to embed our algorithms within a tabu search framework, our algorithms need to identify moves with minimum positive cost if the Procedure Shortest Path fails to find a feasible improving path. Hence we modify the improvement graphs for the CWIM neighborhoods. For each city j 6= s and j 6= 0m+1 we create arcs (s, j 0 ) and eliminate arcs (s, j), and we create arcs (j, 0m+1 ) and eliminate arcs (j 0 , 0m+1 ). These arcs correspond to null moves and have 0 cost; however, they do force the Procedure Shortest Path to find a path that corresponds to at least one non-null move. 4 Computational Study for the VRP In this section we provide the results of a computational study of the CIM neighborhood search algorithm embedded into a tabu search framework. In our computational experiments we primarily use two sets of standard test problems for the VRP and the VRP with distance constraints (DVRP). The 14 small instances described in [5] are obtained from the ORLIB at http://mscmga.ms.ic.ac.uk/jeb/orlib/vrpinfo.html. The rest of the instances that are larger in size are described in [19]. We compare the results obtained on these instances by our large scale neighborhood search algorithm to the results reported in [19], [34], and [25] which was published during the initial review process of this paper. 4.1 Computational Framework and Parameter Settings The experimental design framework and the values of the parameters we used for obtaining the results throughout this paper are as follows: 21 1. To solve problems with a given fleet size, we let our algorithms create capacity and distance infeasible solutions but penalize the amount of infeasibility. To do this, we use a common method (see [15] and [34]) where the cost of a move is modified to be equal to the change in total distance plus a constant times the change in infeasibility. This constant is set to 1 initially and then (while making sure that it stays within the range (1, 2500)) is increased (multiplied by two) or decreased (divided by two) after creating 10 consecutive infeasible or feasible solutions, respectively. 2. We use a randomized version of the Clark and Wright (CW) algorithm [6] to construct our initial solutions. (Experimentation with algorithms which generate higher quality initial solutions did not provide better final results.) If the solution generated uses more vehicles than the fleet size, we eliminate the extra vehicles with the largest spare capacities by inserting the customers they serve into the tours of the other vehicles without considering the capacity and distance constraints. A 2-opt procedure on each vehicle’s route is used at the end. 3. After the initial solution is created, we search for a compounded move by constructing an improvement graph and finding a shortest path on this graph. We search for the best compounded weakly independent moves based on the ST, PO, and MCS improvement graphs of Section 3. At each iteration we search a CWIM neighborhood based on one of the three improvement graphs selected randomly. Next we give some particulars of the search based on each improvement graph: (a) For the ST-improvement graph, the ordering of the vehicles affects the graph and this affects the neighborhood searched. In our heuristic, at each iteration, we create a random order of the vehicles. (b) For the PO-improvement graph, we consider an improvement graph based on the current −1 ) be a routing where the cities routing T = (T1 , T2 , ..., Tm ). Let T −1 = (T1−1 , T2−1 , ..., Tm in each tour Tk−1 are visited in exactly the opposite order of the tour Tk . Assuming the distances between cities are symmetric, if routing T is feasible then routing T −1 is also feasible. We consider an alternative PO-improvement graph based on the routing T −1 . In our heuristic, four CWIM neighborhoods based on these two PO-improvement graphs are generated and searched for improving solutions. 22 (c) For the MCS-improvement graph, we allow for swapping a set of consecutive cities. At each iteration, we allow for swapping sets with cardinality of at most S and we further restrict the search process by only allowing swaps between sets of the same cardinality. The computational results presented in this paper are obtained by choosing S randomly between 1 and 3. 4. If the shortest path in the current improvement graph has a negative cost, then the move implied by this shortest path is applied to the current solution, and the improvement graph is updated for the next iteration. If the shortest path found has a non-negative cost, then our heuristic was unable to find an improved solution. Nevertheless, we incorporate some of the exchanges implied by the positive cost shortest path found. 5. Since moves with positive costs are allowed, in order to prevent cycling, we embed the neighborhood search heuristic into a very simple tabu search mechanism (see [18]). If a node is moved from its current location in the latest compounded move, it is made tabu for z iterations of the algorithm. When a node is made tabu it is not allowed to be moved in an exchange by the elimination of its corresponding improvement graph arcs. After each move, the value for z is chosen at random from the interval [zmin , zmax ]. For generating the computational results in this paper, we let [zmin , zmax ] = [5, 12]. 6. When the search starts to stagnate, that is, when no improving solutions are generated for a while, we diversify the search by generating a larger sized neighborhood, MCS-all, based on the MCS-improvement graph. MCS-all neighborhood allows for swapping sets of cities with cardinality in the range [min−set, max−set] as long as these cities are visited by vehicles that are close to each other. We say that two vehicles are close if the distance between the average coordinates of all the cities on the vehicles is less than α times the average distance over all the vehicles. The computational results presented in this paper are obtained by choosing α randomly between 0.3 and 0.9. Values of min− and max − set are calculated dynamically n by choosing min − set randomly in the range (3, max 4 ) where max n is the number of cities on the vehicle visiting the most cities. max − set is always set to min − set + 15. In our implementation we use the following diversification strategies: (a) If no improving solutions are found for the last 20 iterations then the MCS-all neighborhood is searched with probability 2/300 instead of one of the six original neighborhoods 23 described above. (b) If no improving solutions are generated for 60000/n iterations then k ∗ √ n/3 MCS-all neighborhoods are searched in a row. Here n is set to be equal to the number of cities, and k is the number of MCS-all neighborhoods searched without generating an improving solution. √ (c) If k ∗ n/3 MCS-all neighborhoods are created and searched as above for 4 times then we perform 5 random moves within the MCS-all neighborhood of the best solution found so far. This set of random moves helps to restart the search at a solution which is sufficiently different than the current best solution and computationally performed better than restarting the search with a new random solution. 7. Finally, anytime after we use a diversification move or find a new improving solution, we use a 2-opt procedure on each vehicle’s tour. 4.2 Computational Results In this section we briefly discuss several competitive heuristics and their computational settings. Taburoute (TR) developed by Gendreau et al. in [15] is based on single insertion moves. Given a node, TR uses a sophisticated methodology to choose a good tour for it to be inserted into. Two other features of the algorithm are that it allows for infeasible solutions to be searched and that periodically the tour in which a vertex has just been inserted in is re-optimized. The times for TR are given for computations on a Silicon Graphics workstation, 36 MHz, 5.7 Mflops. Taillard’s tabu search algorithm (TA) in [31] and the algorithm by Rochat and Taillard [30] (RT) produce some of the best results for the 14 problems in [5]. TA uses both insertion and swap moves. Also, only the creation of feasible tours are allowed, and occasionally individual tours are re-optimized. A novel feature of the algorithm is that it is suitable for decomposition into subproblems. Hence Taillard uses parallel computing to generate solutions and the overall computation times are not reported. However on average it takes 11.58 and 21.73 cpu minutes to reach solutions with objective functions within 5% and 1% of the best known values. These computations are conducted on a Silicon Graphics workstation with 4.3 Mflops. The running times for RT also are reported in a similar fashion. We also report the results from a parallel implementation by Rego and Roucairol [28] of an ejection chain based tabu search algorithm (RR). The times for the algorithm are given for computations on a Sun Sparc4 IPC, 2.5 - 4.3 Mflops. 24 The results from a tabu search algorithm, based on applying a few simple swaps and insertion moves to the current solution, developed by Xu and Kelly (XK) are obtained from [19]. The XK results are based on computations performed on a DEC Alpha computer with 26-43 Mflops. The results of a deterministic annealing algorithm called Record-To-Record Travel (RTR) reported in [19] are also presented for the large scale instances obtained from [19]. RTR was run on a Pentium 100 MHz PC with 12 Mflops. The results by Toth and Vigo [34] on their Granular Tabu Search (GTS) algorithm is also reported. The GTS approach is based on using smaller neighborhoods that include moves that are more likely to lead to improved solutions. The GTS results are based on computations on a Pentium 200 MHz PC with 15 Mflops. Finally we report the recent results of Prins’ [25] hybrid genetic algorithm (GA). The results we report on GA are obtained on a 1 GHz Pentium III PC with 75 MFlops. In our implementation of the CIM algorithm, to get the average values and times reported, we use a standard set of parameters and take the average over 5 runs of length 6000 seconds. We choose 6000 seconds as a stoping criteria because in our initial experimentations, we have concluded that approximately 300,000 neighborhoods for a 200 node problem are searched in this time frame and very few improvements are obtained if the algorithms are run longer. The results we report are obtained on a SunBlade100 workstation with a 500 MHz CPU UltraSparcIIe processor. According to [11] this CPU has similar speed as the CPU used by Prins. In Table 1, we give a summary of the CPUs used to obtain the results for the heuristics above. The last column, ρ, gives a rough conversion factor relative to a 1 GHz Pentium III PC with 75 MFlops. In Table 2, we summarize our benchmark instances. Following the standard convention, we denote the test instances by a name of the form tnnn − kk, where t is the instance type, equal to E for VRPs and to D for DVRPs, and nnn and kk are the number of nodes and available vehicles respectively. Table 2 presents the previous best known solution value, the solution source, and the best solution values we have obtained through out our experimentation. We were able to improve the best known solutions for 8 of the large scale VRP instances and 1 large scale DVRP instance. The value of our best solution is written in bold if it improves the previous best value. From Table 2, we conclude that TA is the most effective algorithm for the 14 small instances, and CIM and GA dominates the other algorithms on large VRP and DVRP instances, respectively. Next in Tables 3 and 5, we compare the behavior of the CIM algorithm in terms of solution 25 quality and computing time with the algorithms mentioned above. Although TA and RT algorithms have determined the best known solutions for the 14 classic instances, we do not include their results in Tables 3 and 5 because their overall computing times are not reported. However, in Table 4 we compare these algorithms to CIM, GTS, and XK in terms of the time required to reach a prescribed solution quality. Table 3 compares the performance of CIM, GA, GTS, TR, XK and RR on the fourteen classic VRP and DVRP instances. For each instance the table reports the percentage deviation of the solution values obtained by each algorithm from the best known solution values and the computing time in minutes on different CPUs. The last line of the table scales the overall computing time on different CPUs using the conversion factor ρ given in Table 1. On these fourteen instances, CIM generated solutions with the second minimum overall deviation from the best known solution values. It performs comparatively better on larger instances. Although GA generates better quality solutions overall, its computing time is an order of magnitude higher than that of CIM. GTS and RR algorithms both create good solutions in an extremely small amount of computing time. On the VRP instances, XK algorithm gives the best quality solutions; however, this does not carry over to the DVRP instances. In Table 4, we present the time to reach 1% and 5% of best known solution values on the 7 classic VRP instances from [5] for CIM, GTS, T, RT, and XK. Note that the best known solution values used in each paper to determine the percentage deviation could be slightly different for each algorithm. As also observed in [34], all of the tabu search algorithms previously considered show a similar behavior where a solution within 5% of the best known value is reached in a few seconds and reaching a solution with 1% deviation takes 10 − 25% of the overall computing time. On average, CIM finds a solution within 5% of the best known value in less than a second. When scaled times are considered it is only slower than GTS. However, CIM is slower in reaching better quality solutions and requires about 50% of the overall computing time to obtain solutions with 1% deviation of best known solution values. Finally in Table 5, we report the performance of the CIM heuristic on the 20 large-scale instances, introduced in [19]. We compare the average percentage deviation and computational time of CIM heuristic, obtained over 5 runs, with the results of GA, GTS, XK, and with a deterministic heuristic, RTR, described in [19]. The results of the XK algorithm on the capacity and distance restricted VRP are not presented because the results reported in [19] are obtained with an incorrect distance matrix. CIM heuristic not only improves the previously best known solution values for 26 8 of 12 VRP instances but also on average has the best performance in terms of solution quality. GA is the second best algorithm in this category followed by GTS, RTR, and XK. For the DVRP instances, CIM improves the previously best known solution value for 1 of 8 instances. On average, the GA heuristic performs better in this category. GTS and RTR heuristics perform significantly worse than CIM and GA with more than 3% deviation. Considering the entire collection of instances, CIM and GA heuristics have very similar average deviations and running times. Their average deviations are significantly smaller than that of GTS and RTR, but their speed is an order of magnitude slower. 5 Summary We have demonstrated a systematic way of creating and searching compounded independent moves neighborhoods for routing problems. The main contribution of the paper is to extend these very large scale neighborhood search ideas to problems with side constraints. For such problems, the exact search of the CIM neighborhood becomes NP-hard. Hence, we have introduced a multi-label shortest path algorithm for searching these neighborhoods heuristically. Next we performed a computational study on the vehicle routing problem with capacity and distance restrictions. Our experiments on the classic fourteen test problems and the twenty largescale instances show that the compounded moves neighborhoods are competitive approaches for solving vehicle routing problems. In fact, CIM algorithm was able to improve the best known solution values for 9 of the 20 large-scale instances. Acknowledgments Özlem Ergun was supported in part by NSF contract DMI-0238815. James Orlin was supported in part by NSF contract DMI-0217123. 27 References [1] E. Aarts and J.K. Lenstra, Local Search in Combinatorial Optimization, (John Wiley & Sons, New York, 1997). [2] R.K. Ahuja, Ö. Ergun, J.B. Orlin, and A.B. Punnen, A survey of very large-scale neighborhood search techniques, Discrete Applied Mathematics 123 (2002) 75-102. [3] R.K. Ahuja, J.B. Orlin, and Thomas L. Magnanti, Network Flows, (Prentice-Hall, NJ, 1993). [4] R.K. Ahuja, J.B. Orlin, and D. Sharma, Multi-exchange neighborhood search algorithms for the capacitated minimum spanning tree problem, Mathematical Programming 91 (2001) 71-97. [5] N. Christofides, A. Mingozzi, and P. Toth, The vehicle routing problem. N. Christofides, A. Mingozzi, P. Toth, and C. Sandi, eds., Combinatorial Optimization, (Wiley, Chichester, 1979). [6] G. Clarke and J.W. Wright, Scheduling of vehicles from a central depot to a number of delivery points, Operations Research 12 (1964) 568-581. [7] R.K. Congram, Pollynomially searchable exponential neighborhoods for sequencing problems in combinatorial optimization, Faculty of Mathematical Studies Thesis, University of Southampton, 2000. [8] R.K. Congram, C.N. Potts, S. L. van de Velde, An iterated dynasearch algorithm for the single machine total weighted tardiness scheduling problem, INFORMS Journal on Computing 14 (2002) 52-67. [9] V. Deineko and G.J. Woeginger, A study of exponential neighborhoods for the traveling salesman problem and the quadratic assignment problem, Mathematical Programming Series A 87 (2000) 519-542. [10] M. Desrochers and F. Soumis, A generalized permanent labeling algorithm for the shortest path problem with time windows, INFOR 26 (1988) 191-212. [11] J. Dongarra, Performance of various computers using standard linear equations software, University of Tennessee, Knoxville TN, 37996, Computer Science Technical Report Number CS 89 85 (2004). url:http://www.netlib.org/benchmark/performance.ps 28 [12] Ö. Ergun, New neighborhood search algorithms based on exponentially large neighborhoods, Operations Research Center Thesis, MIT, 2001. [13] R.T. Firla, B. Spille and R. Weismantel, personal communication. [14] M. Fisher, Vehicle routing. M.O. Ball, T.L. Magnanti, C.L. Monma, and G.L. Nemhauser, eds., Network Routing, Handbooks in OR & MS Vol. 8, (Elsevier, Amsterdam, 1995). [15] M. Gendreau, A. Hertz, and G. Laporte, A tabu search heuristic for the vehicle routing problem, Management Science 40 (1994) 1276-1290. [16] M. Gendreau, G. Laporte, and J.-Y. Potvin, Vehicle routing: modern heuristics, E. Aarts and J.K. Lenstra eds., Local Search in Combinatorial Optimization, (John Wiley & Sons, New York, 1997) 311-336. [17] F. Glover, Ejection chains, reference structures, and alternating path algorithms for the traveling salesman problem, Research Report, University of Colorado-Boulder, Graduate School of Business, 1992. {A short version appeared in Discrete Applied Mathematics 65 (1996) 223253.} [18] F. Glover and M. Laguna, Tabu Search, (Kluwer Academic Publisher, Boston 1997). [19] B.L. Golden, E.A. Wasil, J.P. Kelly, and I-M. Chao, Metaheuristics in vehicle routing, T.G Crainic and C. Laporte, eds, Fleet Management and Logistics, (Kluwer Academic Publisher, Boston, 1998) 33-56. [20] R.F. Kirby and R.B. Potts, The minimum route problem for networks with turn penalties and prohibitions, Transportation Research 3 (1969) 397-408. [21] G. Laporte and I.H. Osman, Routing problems: a bibliography, in Freight Transportation, G .Laporte and M. Gendreau, eds., Annals of Operations Research 61 (1995) 227-262 [22] C.L. Liu, Elements of Discrete Mathematics, (McGraw-Hill, Singapore, 1985). [23] I.H. Osman, Metastrategy simulated annealing search algorithms for the vehicle routing problem, Annals of Operations Research 41 (1993) 421-451. [24] C.N. Potts and S.L. van de Velde, Dynasearch - Iterative local improvement by dynamic programming: Part I, The traveling salesman problem, Technical Report, University of Twente, The Netherlands, 1995. 29 [25] C. Prins, A simple and effective evolutionary algorithm for the vehicle routing problem, Computers and Operations Research 31 (2004) 1985-2002. [26] A.P. Punnen and F. Glover, Ejection chains with combinatorial leverage for the TSP, Research Report, University of Colorado-Boulder, 1996. [27] T.K. Ralphs, L. Kopman, W.R. Pulleyblank, and L.E. Trotter, Jr., On the capacitated vehicle routing problem, Research Report (2000). Updates available at http://www.branchandcut.org. [28] C. Rego and C. Roucairol, A parallel tabu search algorithm using ejection chains for the vehicle routing problem. In I.H. Osman and J.P. Kelly eds., Meta-Heuristics: Theory & Applications, (Kluwer Academic Publisher, Boston, MA, 1996) 661-675 [29] C. Rego, A subpath ejection method for the vehicle routing problem, Management Science 44 (1998) 1447-1459. [30] Y. Rochat and E. Taillard, Probabilistic diversification and intensification in local search for vehicle routing, Journal of Heuristics 1 (1995) 147-167. [31] E. Taillard, Parallel iterative search methods for vehicle routing problem, Networks 23 (1993) 661-673. [32] P.M. Thompson and J.B. Orlin, The theory of cyclic transfers, Operations Research Center Working Paper, MIT, August 1989. [33] P.M. Thompson and H.N. Psaraftis, Cyclic transfer algorithms for multivehicle routing and scheduling problems, Operations Research 41 (1993). [34] P. Toth and D. Vigo, The granular tabu search and its application to the vehicle routing problem,INFORMS Journal on Computing 15 (2003) 333-346. [35] P. Toth and D. Vigo, Vehicle Routing Problem, (Society for Industrial and Applied Mathematics, Philadelphia, PA, 2002). 30 Reference TA [31] RT [30] TR [15] RR [28] XK [19] RTR [19] GTS [34] GA [25] CIM CPU Used SG 4D/35 SG Indigo 100MHz SG 36 MHz Sun Sparc4 IPC DEC Alpha Pentium 100MHz Pentium 200 MHz Pentium 1 GHz Sun Sparc IIe 500 MHz Mflops 4.3 15 5.7 2.5 - 4.3 26.0-43.0 12 15 75 75 ρ 0.06 0.20 0.08 0.05 0.40 0.16 0.20 1 1 Table 1: Summary of the CPUs used in testing various competitive heuristics and rough conversion factors (ρ) relative to a 1 GHz Pentium III PC with 75 MFlops. 31 j i-1 j+1 i Figure 1: Illustrating a single 2-opt move. Dashed lines are deleted from T. Bold lines are added to obtain T’. 32 i-1 j i l+1 j+1 k k-1 l Figure 2: Illustrating two independent 2-opt moves. Dashed lines are deleted from T. Bold lines are added to obtain T’. 33 1 2 3 4 5 6 7 8 9 10 11 12 Figure 3: Let tour T = (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 1). After applying the CI 2-opt moves implied by the path 1 − 2 − 3 − 6 − 11 − 12 tour T 0 = (1, 2, 3, 5, 4, 6, 10, 9, 8, 7, 11, 1) is obtained. 34 1 2 3 4 5 6 7 8 9 10 2’ 3’ 4’ 5’ 6’ 7’ 8’ 9’ 10’ 11 12 Figure 4: Let tour T = (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 1). After applying the CWI insertion moves implied by the path 1 − 2 − 5 − 6 − 90 − 11 − 12 tour T 0 = (1, 2, 4, 3, 5, 6, 8, 7, 10, 9, 11, 1) is obtained. 35 s 01 1 2 3 4 02 5 T1 6 7 03 T2 (a) T1 01 1 3 5 s T2 7 03 02 2 4 6 (b) Figure 5: Let a feasible VRP solution be represented by two tours T1 = (1, 2, 3, 4) and T2 = (5, 6, 7). (a) The solution is single tour ordered. A partial ST-improvement graph for the CI insertion neighborhood is demonstrated with all the arcs that are adjacent to node 1. (b) Before a POimprovement graph is created, the nodes are relabelled so that they are position ordered. A partial PO-improvement graph for the CI insertion neighborhood is demonstrated with all the arcs that are adjacent to node 1. 36 Instance E051-05 E076-10 E101-08 E101-10 E121-07 E151-12 E199-17 E241-22 E253-27 E256-14 E301-28 E321-30 E324-16 E361-33 E397-34 E400-18 E421-41 E481-38 E484-19 Prev. Best 524.61 835.26 826.14 819.56 1042.11 1028.42 1291.45 709.90 868.70 587.09 1014.82 1086.24 746.56 1376.49 1363.34 932.68 1846.55 1650.42 1136.05 Source [15] TA [31] TA [31] and [15] TA [31] TA [31] TA [31] RT [30] GTS [34] GTS [34] RTR [19] GTS [34] GA [25] XK [19] GA [25] GTS [34] XK [19] GA [25] GTS [34] GTS [34] CIM Best 524.61 835.43 826.14 819.56 1042.11 1033.01 1303.21 709.66 869.26 586.44 1010.70 1092.29 745.26 1385.84 1357.97 922.09 1854.54 1642.92 1121.15 % 100.00 100.02 100.00 100.00 100.00 100.45 100.91 99.97 100.06 99.89 99.59 100.56 99.83 100.68 99.54 98.86 100.43 99.55 98.59 D051-06 D076-11 D101-09 D101-11 D121-11 D151-14 D200-18 D201-05 D241-10 D281-08 D321-10 D361-09 D401-10 D441-11 D481-12 555.43 909.68 865.94 866.37 1541.14 1162.55 1395.85 6460.98 5646.63 8412.80 8447.92 10195.59 11036.22 11828.78 13624.52 TA [31], [15] TA [31] TA [31], [15] [23] TA [31] TA [31] RT [30] GA [25] GA [25] GA [25] GA [25] GA [25] GA [25] GA [25] GA [25] 555.43 913.32 865.94 866.37 1543.23 1163.09 1410.13 6460.98 5641.94 8413.82 8472.27 10261.92 11141.25 11875.18 13747.99 100.00 100.04 100.00 100.00 100.14 100.05 101.02 100.00 99.92 100.01 100.29 100.65 100.71 100.39 100.91 Table 2: Summary of the benchmark instances. Best solution values obtained by a CIM algorithm and their % deviation from the previous best solutions are given in the last two columns. The CIM values are written in bold if they improve the previous best. 37 38 Best Soln. E050-05 524.61 E075-10 835.26 E100-08 826.14 E100-10 819.56 E120-07 1042.11 E150-12 1028.42 E199-17 1291.45 VRP Average D050-06 555.43 D075-11 909.68 D100-09 865.94 D100-11 866.37 D120-11 1541.14 D150-14 1162.55 D199-18 1395.85 DVRP Average Overall Average Scaled Time Average Instance Time 5.42 33.67 28.01 1.68 16.56 53.38 62.69 28.77 23.34 37.84 42.51 18.69 46.74 36.62 43.65 35.63 32.20 32.20 % 100.00 100.00 100.00 100.00 100.00 100.31 100.68 100.14 100.00 100.29 100.00 100.00 100.12 100.15 101.74 100.33 100.23 Time 0.50 46.36 27.63 2.70 17.85 330.11 1146.52 224.52 0.66 85.18 22.41 5.16 626.54 434.90 1609.87 397.82 311.17 311.17 GA GTS % Time 100.00 0.81 100.40 2.21 100.29 2.39 100.00 1.10 100.07 3.18 100.47 4.51 102.08 7.50 100.47 3.10 100.00 0.86 101.21 2.75 100.41 2.90 100.00 1.41 100.28 9.34 100.91 5.67 102.86 9.11 100.81 4.58 100.64 3.84 0.77 % 100.00 100.06 100.40 100.00 103.01 100.75 102.42 100.95 100.00 100.39 100.40 100.00 102.12 101.31 101.62 100.78 100.86 TR Time 6.00 53.80 18.40 16.00 22.20 58.80 90.90 38.01 13.50 54.60 25.60 65.70 59.20 71.00 99.80 55.63 46.82 3.75 Time 29.92 48.80 71.93 56.61 91.23 149.90 272.52 102.99 30.67 102.13 98.15 152.98 201.75 368.37 160.30 131.65 52.66 103.11 103.62 101.72 XK % 100.00 100.00 100.00 100.00 100.00 100.11 100.55 100.09 100.00 106.15 101.78 105.64 105.02 Table 3: Computational comparisons on the classic fourteen instances from [5]. % 100.00 100.91 100.02 100.00 100.00 100.55 101.28 100.39 100.00 101.20 100.00 100.00 100.41 100.11 101.13 100.41 100.40 CIM % 100.00 100.27 100.17 100.00 100.14 102.52 103.64 100.96 100.00 100.00 100.27 100.02 100.59 101.40 101.79 100.58 100.77 RR Time 0.85 16.80 33.90 1.22 6.30 16.25 16.25 14.65 3.17 23.10 8.60 9.42 2.00 15.55 52.02 16.26 15.45 0.77 39 CIM 5% 1% 0.02 0.08 0.05 23.85 0.00 0.08 0.00 0.00 0.13 0.13 0.20 23.77 0.32 65.98 0.10 16.27 0.10 16.27 GTS 5% 1% 0.00 0.02 0.00 0.79 0.02 1.05 0.00 0.01 0.00 0.01 0.05 2.12 0.55 0.09 0.67 0.02 0.13 5% 0.12 0.05 0.20 1.35 76.67 1.43 1.25 11.58 0.69 1% 0.82 0.88 9.67 5.67 63.33 50.00 21.73 1.30 TA 5% 0.05 0.03 0.25 0.85 11.50 0.42 1.48 2.08 0.42 16.19 3.24 1% 0.18 1.13 15.00 5.83 45.00 30.00 RT Table 4: Time to reach a given solution quality, in CPU minutes. E050-05 E075-10 E100-08 E100-10 E120-07 E150-12 E199-17 Average Scaled Time Average Instance 5% 0.65 0.05 0.15 0.09 0.52 4.09 6.22 1.68 0.67 1% 0.77 6.72 2.38 2.39 1.18 38.41 201.18 36.15 14.46 XK 40 Best Soln. % 100.23 100.35 100.11 100.01 100.91 99.97 101.10 99.80 99.44 101.01 99.76 99.14 100.15 100.02 100.12 100.61 100.45 101.17 101.16 101.03 101.99 100.82 100.49 Time 50.88 50.58 49.40 44.85 57.40 80.73 84.84 68.82 60.33 76.73 70.09 73.76 64.03 41.24 27.11 47.90 68.21 75.51 83.56 86.30 63.16 61.62 62.83 62.83 CIM % 101.01 101.04 101.67 100.39 100.34 100.65 100.66 100.54 100.76 100.00 100.03 101.38 100.71 100.00 100.02 100.00 100.14 100.71 100.00 100.31 100.77 100.24 100.47 Time 5.86 15.30 14.32 39.33 34.07 36.58 74.25 110.48 78.50 210.42 130.97 30.87 65.08 1.04 32.42 9.97 77.92 39.05 120.83 88.30 187.60 69.64 67.36 67.36 GA GTS % Time 100.16 14.29 100.01 11.43 101.07 11.67 100.20 21.45 100.92 14.51 100.68 15.83 101.78 30.06 100.38 18.45 100.36 33.12 103.75 43.05 100.12 23.07 100.88 42.90 100.86 23.32 103.66 2.38 101.59 4.98 106.54 4.65 101.24 8.28 103.45 11.66 103.32 12.94 101.75 11.08 109.44 15.13 103.88 8.89 102.37 16.10 3.22 % 105.26 101.42 100.34 105.10 102.93 100.00 101.93 100.99 100.00 104.79 100.38 100.31 101.96 Time 2314.00 1465.77 340.20 4101.02 1577.30 501.82 5718.38 4340.07 852.72 10839.73 8943.45 1151.10 3512.13 591.40 802.87 913.70 898.53 1062.73 1749.27 1586.20 2432.42 1254.64 2609.13 1043.65 XK Table 5: Computational comparisons on the twenty large-scale instances. E240-22 709.90 E252-27 868.70 E255-14 587.09 E300-28 1014.82 E320-30 1086.24 E323-16 746.56 E360-33 1376.49 E396-34 1364.23 E400-18 932.68 E420-41 1846.55 E480-38 1650.42 E483-19 1137.18 VRP Average D200-05 6460.98 D240-10 5646.63 D280-08 8412.80 D320-10 8447.92 D360-09 10195.59 D400-10 11036.22 D440-11 11828.78 D480-12 13624.52 DVRP Average Overall Average Scaled Time Average Instance RTR % Time 101.48 5.69 101.42 6.01 100.00 23.01 101.42 8.15 101.61 21.83 100.35 31.49 104.31 12.42 100.00 32.62 100.18 69.19 101.55 31.05 100.46 47.55 100.00 101.09 101.06 32.51 103.74 11.24 103.33 3.68 107.18 18.79 106.56 22.66 109.98 22.55 107.65 40.04 105.79 111.37 107.45 122.61 106.46 44.12 103.76 37.15 5.94
© Copyright 2026 Paperzz