12/6/11
VMware: VMware vSphere Blog: Should I defrag m Guest OS?
VMware vSphere Blog (http://blogs.vmware.com/vsphere/)
BEGIN THE JOURNEY TO A PRIVATE CLOUD WITH DATACENTER VIRTUALIZATION
Storage vMotion, Storage DRS and vSphere Replication Interoperability - Revisited (http://blogs.vmware.com/vsphere/2011/09/storage-vmotion-storage-drsand-vsphere-replication-interoperability-revisited.html)
Main (http://blogs.vmware.com/vsphere/) vSphere Storage Appliance (VSA) Resilience - Network Outage
Scenario #1: Back End (http://blogs.vmware.com/vsphere/2011/09/vsphere-storage-appliance-vsa-network-outage-scenario-1-back-end.html)
09/20/2011
59
Like
12
Communities
Share
8
1
(http://communities.v mware.com)
Should I defrag my Guest OS?
(http://blogs.vmware.com/vsphere/2011/09/should-i-defrag-my-guest-os.html)
Yep - this old chestnut. :-)
This has come up time and time again, and I am going to share with you some conversations that have been occurring within
VMware on this topic. In fact, we've been having these conversations for a long time now.
What is it that defragmentation is supposed to give ou?
Well, historically, if you ran a defragmentation operation against an OS disk (typically Windows), you would expect to see a
performance improvement. Defragmentation moves blocks around the disk to bring together blocks belonging to the same file
in an effort to make the file contiguous on disk. This means that sequential I/O operations should be faster after a defrag.
Here's a view of the Disk Fragementer that is part of the System Tools with Windows 7:
(http://blogs.vmware.com/.a/6a00d8341c328153ef014e8bb247da970d-pi)
What about defragmentation of a Guest OS in a Virtual Machine?
This is very different to running a defrag on a physical host with a local disk. Typically you are going to have multiple VMs
running together on a VMFS or NFS volume. Therefore the overall I/O to the underlying LUN is going to be random so
defragmenting individual Guest OS'es is not really going to help performance. However, there are other concerns that you need
to keep in mind. The easiest way to explain the concerns is to give you some scenarios of what might happen to a VM which is
defraged, and what impact it has on the various vSphere technologies. You can then make up you own mind about whether it is
a good idea or not.
1. Thin Provisioned VMs. If you defragment a Thin Provisioned VM, as file blocks are moved around, the TP VMDK bloats
up, consuming much more disk space.
2. Linked Clone VMs (vCloud Director, View). In the case of a VM running off of a linked clone, the defragmenter bloats up
the linked clone redo logs.
3. Replicated VMs (Site Recover Manager, vSphere Replicator). If your VM was being replicated, and you
defragemented the VM on the protected site, it could well cause a lot of data to be sent over the WAN to the replicated
site.
4. Snapshot'ed VMs. This is a similar use case to Linked Clones. Any VMs running off of a snapshot which ran a defrag
would cause the snapshot to inflate considerably, depending on how many blocks were moved during the defrag
operation.
5. Change Block Tracking (VMware Data Recover ). The CBT feature is used heavily by backup products, including
blogs.vmware.com/vsphere/2011/09/should-i-defrag-m -guest-os.html
1/5
12/6/11
VMware: VMware vSphere Blog: Should I defrag m Guest OS?
VM a e Da a Rec e (VDR). Thi fea
i
d i g a bac
e a i , he
be bac ed , ea i g a
ge bac
e ac cha ge
a VM' di b c d i g a bac
be f b c
ha cha ge
i i c ea e, hich ea
i e.
6. Storage vMotion. S age M i a
e ai
a i i ia ed, i
da
i
d i g he ig a i .
Def ag e a i
a
ge e a e
ef
a ce i
e e
e ha e
b e ed a
NAS ba ed da a
Ia
a
e CBT i S he e 4.0. If a VM a bei g S age M i 'ed he a def ag
ac he i e c
e e he e a i
i ce he def ag i cha gi g b c
e I/O
he di
ha
igh be gai ed f
iceab e i
e e i
. Thi
c
d be
e
fa c
ce
c
e
ha
a
ib e
he def ag. I h d
i
ha I ha e ead ha , i e a a VM a e,
ef
a ce af e a def ag e a i
f G e OSe e idi g
SAN
e .
high igh a addi i
age a a i
def ag e a i
e a i . If a def ag
e da a i ha e
ca ab e f
f he G e
a
ce a i
ha
e a a a ba ed ech
i g b c
f da a be
OS d e ' ea
a e
ee diffe e
ch e e. If
g
a he ha a S he e ech
g . If
age ie (SSD/SAS/SATA), e.g. EMC FAST, he
VM ha bee
i g f
e i e
ie ed
age, he i a i e ih d he a a ha a ead ea
he e he h -b c a e, a d ha e ca ed he e
g ahead a d def ag, a d
e a f he VM' b c a
d agai , he a a i g i g
ha e
e ea
h -
he SSD. If
he e he
a e.
(h ://b g .
If
a
ef
a e.c
/.a/6a00d8341c328153ef015391be9433970b- i)
a e he def ag
a ce gai if he VM i
O e ai gS
e
de
eg a , I hi
hi c d ca e a e f
ed
a da a
e bac ed b ie ed
a ce dec ea e a he ha gi e
a
age. Thi
a a ead be e ab ed
f
e
.
What do the Storage Arra vendors sa ?
Ne A
ha e a e
g
d S he e/Ne A
i e
e abi i
WP
(h :// edia. e a .c
/d c
e
/ -3749. df)
i
hich he b ief di c
hi
ic. Q i g di ec f
he a e - "VMs stored on NetApp storage arra s should not use disk defragmentation utilities
b ecause the WAFL file s stem is designed to optimall place and access data at a level b elow the guest operating s stem
(GOS) file s stem. If a software vendor advises ou to run disk defragmentation utilities inside of a VM, contact the NetApp
Glob al Support Center b efore initiating this activit ."
What do ou recommend?
M ec
h d
e da i i not
ha e a
be
Ge
ifica i
(h :// i e .c
e a def ag
i
e i
ai e ab
f
f he e b g
/#%21/VM a eS
i g
he G e OS. If
ib e
c
a d
e
e VM a e S
a e bei g ad i ed
i g he c e i
age i f
ai
b
e a def ag e a i
hi b g
i g.
f
i g
e
,
T i e:
age) VMwareStorage
Posted by Chogan on September 20, 2011 in VMw are Storage (http://blogs.vmware.com/vsphere/storage/) , VMw are vSphere
Permalink (http://blogs.vmware.com/vsphere/2011/09/should-i-defrag-my-guest-os.html)
(http://blogs.vmware.com/vsphere/vsphere/)
Comments
(h ://b g .
feed (h
a e.c
://b g .
Whi e ge e a
/
a e.c
he e/2011/09/ h
/
I
d-i-def ag-
he e/2011/09/ h
d ag ee, I
-g e -
d-i-def ag-
/c
-g e -
d ee i
e
/c
/a
.
e
/a
ei
)Y
ca f
)f
.
hi c
hi
e ca e
e ai
b
b c ibi g
he c
e
.
he chec i g hic -
i i
ed fi e e e
e
he
a e eei g f ag e a i
a gi e
e
hi g 120+%. A ea
de Wi d
Se e 2008/2003/200 he e e e
i c ea ed e
ad a d ge e a e f
a ce i
e ca ed f
he f ag e a i . The e affec
e e ee ac
da ace e
ih
i e 10gig i
he e
ide a d 4- a
highe de i 8GB FC SAN c
ec i i
ead
ac
ed ha ed AMS2500 di . I ge e a if f ag e a i i
de 20%, he e i
eed f i i a VM e i
e ....b
he
he e i a high e ce age f f ag e a i
Posted by: Patrick
d e i
i h d
e?
09/20/2011 at 06:16 AM (http://blogs.vmware.com/vsphere/2011/09/should-i-defrag-my-guest-os.html?cid=6a00d8341c328153ef014e8bb2c439970d#comment-
6a00d8341c328153ef014e8bb2c439970d)
Hi Pa ic , ha
f
hic i i ed,
d
'
he c
- a
e
h
. I'
ed VM
e he e a e a a g i g be c
e ca e . Def ag
ca
age. B i a
he c fig a i , e ecia
igh he if
ha e a i g e
f SAN & NAS da a
e ,I
ee he be efi .
Posted by: Chogan (http://profile.typepad.com/chogan1)
09/20/2011 at 06:25 AM (http://blogs.vmware.com/vsphere/2011/09/should-i-defrag-my-guest-os.html?
cid=6a00d8341c328153ef015391bf195c970b#comment-6a00d8341c328153ef015391bf195c970b)
Wha ab
def ag e i g he VMFS
Posted by: Dan Hayw ard (http://www.spug.co.uk)
e i e f? W
d hi ha e e f
a ce be efi ?
09/20/2011 at 02:46 PM (http://blogs.vmware.com/vsphere/2011/09/should-i-defrag-my-guest-os.html?
cid=6a00d8341c328153ef014e8bb55755970d#comment-6a00d8341c328153ef014e8bb55755970d)
Hi Da . VMFS f age e a i
h
d ' be a c
Posted by: Chogan (http://profile.typepad.com/chogan1)
ce . P ea e ee h
:// b.
a e.c
/ b/1006810. (h
:// b.
a e.c
/ b/1006810.)
09/20/2011 at 11:44 PM (http://blogs.vmware.com/vsphere/2011/09/should-i-defrag-my-guest-os.html?
blogs.vmware.com/vsphere/2011/09/should-i-defrag-m -guest-os.html
2/5
12/6/11
VMware: VMware vSphere Blog: Should I defrag m Guest OS?
cid=6a00d8341c328153ef014e8bb76679970d#comment-6a00d8341c328153ef014e8bb76679970d)
Thanks a lot, I've been through this debate with me time. Now I have something written to back my point of view.
Posted by: Ashraf Al-Dabbas
09/21/2011 at 02:22 PM (http://blogs.vmware.com/vsphere/2011/09/should-i-defrag-my-guest-os.html?
cid=6a00d8341c328153ef014e8bbb30a2970d#comment-6a00d8341c328153ef014e8bbb30a2970d)
Hi Chogan, I'm the Product Manager at a VMware Elite TAP partner and the makers of a VM Ready certified optimizer for ESX/i.
As you noted, there the side effects that traditional (legacy) defrag can cause, and I appreciate your bringing attention to this.
However, there are solutions on the market (not the Windows defragmenter though) that have technology that mitigates or
entirely eliminates those undesirable effects. Those third party solutions exist and flourish as there are tangible benefits from
reducing fragmentation, including on SANs. I can provide you this data. Please email me to discuss further.
Posted by: Michael Materie (http://www.diskeeper.com)
09/22/2011 at 12:46 PM (http://blogs.vmware.com/vsphere/2011/09/should-i-defrag-my-guest-os.html?
cid=6a00d8341c328153ef015435a09f26970c#comment-6a00d8341c328153ef015435a09f26970c)
Hi Michael,
If there is a product on the market that addresses the above issues with the defragmentation of a Guest OS, then I'd love to
read about it.
Will reach out shortly for references.
Posted by: Chogan (http://profile.typepad.com/chogan1)
09/23/2011 at 12:19 AM (http://blogs.vmware.com/vsphere/2011/09/should-i-defrag-my-guest-os.html?
cid=6a00d8341c328153ef015435a367aa970c#comment-6a00d8341c328153ef015435a367aa970c)
Thanks for writing this. Possible to add this kind of info to our manual? This info deserves more than a blog post.
Posted by: iw an 'e1' ang (http://virtual-red-dot.blogspot.com)
09/23/2011 at 06:24 AM (http://blogs.vmware.com/vsphere/2011/09/should-i-defrag-my-guest-os.html?
cid=6a00d8341c328153ef014e8bc54647970d#comment-6a00d8341c328153ef014e8bc54647970d)
Thanks for commenting Iwan. Let me look into that.
Posted by: Chogan (http://profile.typepad.com/chogan1)
09/23/2011 at 06:36 AM (http://blogs.vmware.com/vsphere/2011/09/should-i-defrag-my-guest-os.html?
cid=6a00d8341c328153ef015391d17ddd970b#comment-6a00d8341c328153ef015391d17ddd970b)
Great post! This is something that I've often thought about too.
I agree with you that most SAN/NAS filesystems are going to abstract the underlying disk layout and thus defragmentation
would not be a beneficial operation (potentially detrimental in fact). Your point about performance is spot on too; you'd be really
lucky to recoup the performance impact of running a defrag against a VM on a shared VMFS volume. However, I would argue
that those using simple DAS or entry level SAN technology (like an HP P2000G3) would still benefit from a guest level defrag
because there isn't any "intelligence" going on in terms of how the controllers are writing data to the disks (excluding the use
cases you brought up above too). Furthermore, I'm still concerned about the implications of a highly fragmented guest
filesystem (even if the guest is completely ignorant of the underlying layout that actually resides on disk), for instance:
- Would a highly fragmented (from the guest OS's point of view) NTFS filesystem's chkdsk speed be negatively affected?
- Would the overall health of a Windows VM with a highly fragmented NTFS filesystem still be OK? Thinking about the affects on
the MFT here too...
Posted by: Ryan
10/08/2011 at 03:05 PM (http://blogs.vmware.com/vsphere/2011/09/should-i-defrag-my-guest-os.html?cid=6a00d8341c328153ef015435fdee24970c#comment-
6a00d8341c328153ef015435fdee24970c)
Hi Ryan,
Thanks for commenting. My goal with this blog post was to highlight some reasons why defrag would not be necessary in a
virtual infrastructure as a whole (or perhaps even a bad idea). However there will always be corner cases, one of which you
mentioned above. If the master file table (MFT) of an NTFS filesystem is also fragrmented (extreme case scenario), then there
will be additional overhead incurred while the OS first retrieves the MFT entry before getting the NTFS data.
A previous poster mentioned the fact that V-locity 3 from Diskeeper is virtualization feature aware and also does optimal file
placement to prevent defrag in the first case. I'm going to check them out at VMworld 2011 in Copenhagen to get further details.
Posted by: Chogan (http://profile.typepad.com/chogan1)
10/10/2011 at 07:29 AM (http://blogs.vmware.com/vsphere/2011/09/should-i-defrag-my-guest-os.html?
cid=6a00d8341c328153ef015392338718970b#comment-6a00d8341c328153ef015392338718970b)
So while I am in agreement with some of the assertions in this article, I feel the conclusion is not complete. One overriding rule
taken from the experience I've had with vmware (primarily storage) is that the entire stack must be optimized for best results or
it will cause stress on the next weak point elsewhere in the stack. Focusing on one part while ignoring another will not yield
best results. I feel that was done in this write up by focusing on the lower layers (SAN, channel, and VMFS), but completely
missed the guest layer!
He is accurate in speaking that as a rule, running defrag against TP (thin provision), LC (linked clone), or auto-tiering, is a bad
idea and should be avoided. However, in the case of systems that are designed at the outset to be a high IO/low latency NTFS
filesystem, TP and LC wouldn't be used, and auto-tiering hasn't been around long enough to employ. Thus, we'll assume in
this conversation we're using a plain-jane thick-provisioned FC disk on a shared VMFS filesystem.
SAN technology abstracts physical disk from the server. This is well known and understood: the ESX doesn't talk to the disks, it
talks to the cache on the frame, thus a defrag operation (take block at location A and move to location B) doesn't really "move"
the block, as the cache deals with that, so defrag will not have any benefits at the SAN layer. Additionally, by its nature, vmware
will always be pure random IO from the frame's perspective, and defrag can't gain us anything there either.
Now the big part that the author failed to look at: how things are from the NTFS point of view in the guest OS. This is a HUGE
consideration. Every file location on NTFS volume is tracked in the MFT (master file table). The MFT is a flat linear file and 1024
bytes is allocated per MFT entry that holds file attributes and extent data which describes each extent that a file sits on in the file
system. An extent in this context is defined as a series of contiguous NTFS clusters (blocks). A contiguous file has one extent
entry, essentially "Starting offset and length". A fragmented file can have many extent entries. Additionally a heavily fragmented
blogs.vmware.com/vsphere/2011/09/should-i-defrag-m -guest-os.html
3/5
12/6/11
VMware: VMware vSphere Blog: Should I defrag m Guest OS?
1024
MFT
.R
I
'
MFT
MFT. N
,
MFT
MFT,
?T
MFT
),
IT
.S
OS
MFT
(
.N
FC
CIO
-
VM IO
VMFS
. :) G
SAN
IO
,
,
,
/
,
I
MFT
.
I
,
'
,
.F
(
), I
.
Posted by: Andrew
10/18/2011 at 07:52 AM (http://blogs.vmware.com/vsphere/2011/09/should-i-defrag-my-guest-os.html?cid=6a00d8341c328153ef01543638b8e6970c#comment-
6a00d8341c328153ef01543638b8e6970c)
H A
F ,
Y
,
. I'
,
'
,
I
')
'
V
.
R
.T
(I
.Y
.T
M
,
.
B ,
,
VM
.A I
,
I'
.
Posted by: Chogan (http://profile.typepad.com/chogan1)
10/22/2011 at 07:00 AM (http://blogs.vmware.com/vsphere/2011/09/should-i-defrag-my-guest-os.html?
cid=6a00d8341c328153ef015392806916970b#comment-6a00d8341c328153ef015392806916970b)
H C
A
,
'
NTFS
.T
VM
O
D
VM
VM
VM
S
'
1. T
I/O
E
NTFS
.
TAP
W
.L
S
'
.W
.
28%.
2. A 12
I/O
3. A 50%
I/O
4. S
30
I/O
50%
5. S
28%.
F
I/O
SCSI
.T
I/O
.
I
/
VM
VM
.
.A
.T
Posted by: Bob Nolan (http://www.raxco.com)
-
10/24/2011 at 08:14 AM (http://blogs.vmware.com/vsphere/2011/09/should-i-defrag-my-guest-os.html?
cid=6a00d8341c328153ef0162fbe1c1ab970d#comment-6a00d8341c328153ef0162fbe1c1ab970d)
H B ,
I
NAS,
.
,
),
. I'
-
I/O ,
(
). T
Posted by: Chogan (http://profile.typepad.com/chogan1)
(SAN,
VM
.
10/26/2011 at 08:00 AM (http://blogs.vmware.com/vsphere/2011/09/should-i-defrag-my-guest-os.html?
cid=6a00d8341c328153ef0162fbee6476970d#comment-6a00d8341c328153ef0162fbee6476970d)
Verif
our Comment
Previewing our Comment
Posted by:
T
.Y
P
Y
Y
.
E
.E
:
.P
(
T
A
:
0;)
.P
,
blogs.vmware.com/vsphere/2011/09/should-i-defrag-m -guest-os.html
.
.T
4/5
12/6/11
VMware: VMware vSphere Blog: Should I defrag m Guest OS?
automated programs from posting comments.
Having trouble reading this image? View an alternate. (#comment-captcha)
Continue
Post a comment
If you have a TypeKey or TypePad account, please Sign In (https://www.typekey.com/t/typekey/login?
v=1.0&t=0ea246f11205a535277c94b1f6ffa83e90f1c920&lang=en_US&_return=http%3A%2F%2Fblogs.vmware.com%2Fvsphere%2F2011%2F09%2Fshould-i-defrag-my-guest-os.html&_portal=typepad) .
Name:
Email Address: (Not displayed with comment.)
URL:
Remember personal info?
Comments:
Preview
Post
Communit Resources
VMworld
Get Support
Other Links
Developer Center
VMw orld.com (http://www.vmworld.com/)
Dow nloads & Patches
Contact Us (http://www.vmware.com/company/contact/)
(http://communities.vmware.com/community/developer/)
VMw orld Session & Labs
(http://www.vmware.com/support/)
About VMw are (http://www.vmware.com/company/)
Documentation
(http://www.vmworld.com/vmworld/sessions.jspa)
Product Support Centers
(http://www.vmware.com/support/pubs/)
(http://www.vmware.com/support/product-support/)
Technical Papers
Manage Licenses
(http://www.vmware.com/resources/techresources/)
(http://www.vmware.com/support/licensing/)
Compatibility Guides
Search the Know ledgeBase
(http://www.vmware.com/resources/guides.html)
(http://kb.vmware.com/)
Jobs at VMw are
(dcr:/templatedata/content/standard/data/sections/other/jobs/index)
Customer Case Studies
(http://www.vmware.com/company/customers/)
Events (http://www.vmware.com/company/events/)
Community Site Map
(http://communities.vmware.com/site-map.jspa)
Copyright
2011 VMw are, Inc. All rights reserved.
blogs.vmware.com/vsphere/2011/09/should-i-defrag-m -guest-os.html
rss feed (http://vmware.simplefeed.net/subscription)
5/5