Datacastle
Storage Configuration Guide
1
Copyright © 2010 Datacastle Corporation. All rights reserved.
Copyright© Datacastle Corporation 2010. All rights reserved.
Datacastle™ is a registered trademark of Datacastle Corporation.
Distribution of substantively modified versions of this document is prohibited without the explicit
permission of the copyright holder.
Distribution of this work or derivative work in any standard (paper) book form for commercial purposes
are prohibited unless prior permission is obtained from the copyright holder.
DOCUMENTATION IS PROVIDED «AS IS» AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS
AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH
DISCLAIMERS ARE HELD TO BE LEGALLY INVALID.
2
Copyright © 2010 Datacastle Corporation. All rights reserved.
Storage configuration
The storage configuration system is designed to cover many scenarios, from enabling use of a number of
existing storage devices, to ensuring that the data is duplicated in to 2 or more locations, to enabli ng
changes to the configuration while the system is running. This document outlines the basic functionality
then describes how to configure the system for some specific scenarios.
Configuration
Storage is configured into Logical Block Partitions which have one or more storage groups, which have
one or more storage locations which map to physical locations.
Storage Location
1A
\\ServerA\Store
Storage Location
1B
\\ServerB\Store
Storage Location
2
\\ServerC\Store
Storage Group 1
Logical Block
Partion
Storage Group 2
Most vaults have a single Logical Block Partition (LBP) defined. Additional partitions would be defined if
there are organizations in the vault that need their data stored separately for some reason. This may be
based on their security requirements, or may be based on the different service levels. For example if
there is an offering that costs more but has higher requirements on data recovery times that mean you
want to store the data in 2 different locations, but only for those customers paying the higher price. The
LBP needs to be configured and the organization (partner or company for example) configured to use it
before any devices are created in that organization.
Within a LBP, one or more storage groups are defined. Datacastle will make sure that it writes the data
to all required storage groups before marking a job as successful.
Within each storage group, one or more locations are defined. Each location has a setting for availability
for reading or writing to enable it to be taken offline for maintenance and weights to determine relative
usage rates.
Settings on the group determine how Datacastle uses the locations. The MinimumStores setting
specifies the number of locations to write to. For example, if there are 3 locations and MinimumStores is
2, Datacastle will write to 2 out of the 3 locations before the write is recorded as successful. It chooses
locations randomly out of those available, taking into account the weighting. For example, if one
location has a weight of 100 and another has weight of 50 and only one location is needed, then about
2/3 of the blocks will go to the first and 1/3 to the second. Weights are available for both reading and
writing.
3
Copyright © 2010 Datacastle Corporation. All rights reserved.
Examples
Adding storage
I created a vault with a single storage location. This location is getting close to full and I can’t easily
increase its capacity, so I purchase another storage device. I add this device as a second location in the
same storage group and leave the MinimumStores setting at 1 so only writing to one location is needed.
Initially I configure both storage locations as available with weight of 50. This means that half the new
data goes to each location.
After a while the original location is as full as I would like it to be, so I reconfigure the location entry to
be unavailable for writing but still available for reading. This means that all new data will be written to
the second location, but a restore can pull data from both locations.
FirstLocation
Logical Block
Partion
Storage Group
MinStore=1
Read only
SecondLocation
Read/Write
\\serverA\FullStore
\\NewServer\Store
Maintaining 2 copies of data
I want to ensure that I have 2 copies of all block data available. In this case if I have a hardware issue
with one of those copies the other will still be available to service restore requests. I know this doesn’t
replace having a backup but it improves the service to my customers because I don’t need to wait for a
backup to be restored to restore service.
I have 3 storage devices - A, B and C. Device A has more storage than the other so I set up one storage
group with a single location on device A and a second storage group with 2 locations, one on device B
and one on device C. The second storage group only needs to write to one location. This means that
every block will be written to device A and also one out of device B or device C.
Write operations such as backup will fail if data storage requirements are not met. This means that if
device A is not available, backups will all fail.
4
Copyright © 2010 Datacastle Corporation. All rights reserved.
Storage Group 1
Logical Block
Partion
Location A
\\serverA\Store
LocationB
\\ServerB\Store
LocationC
\\ServerC\Store
StorageGroup 2
Maintaining a near real-time backup in a remote data center
The ability to write to 2 locations as in the above example enables some additional disaster recovery
scenarios.
If a full data center failure occurs, service can be restored in a different data center if all the data is
available. This means both the files from the storage locations and the database contents. If these are
available, then a new server could be deployed and the data attached and service restored.
Using multiple storage groups can facilitate this by taking care of maintaining an up to date copy of the
block data, but it also introduces some complications. Because Datacastle needs to write to both groups,
if it can’t then backup jobs will fail. If one storage location is accessed over a less reliable link because it
is remote, this can be a problem.
One way to increase the reliability is to have an alternate local location that is only used if the remote
one is not available. For example, if the first storage group above has device A (in a remote data center)
with weight of 100 and device A’ (in the local data center) with weight 0, then Datacastle will always try
to write to device A, but if that fails it will go to device A’.
The diagram below assumes the application server is in WA and that a remote store is available in OR.
The primary storage group is in WA and has a single location. The secondary storage group has a large
storage location in OR, with a smaller backup one in WA (not on the same storage device as the primary
store).
5
Copyright © 2010 Datacastle Corporation. All rights reserved.
PrimaryGroup
\\WAServerA\BigStore
Logical Block Partion
\\ORServer\BigStore
Weight=100
SecondaryGroup
\\WAServerB\SmallStore
Weight=0
Under normal operation the data will be written to \\WAServerA\BigStore and \\ORServer\BigStore. If
the connection to \\ORServer\BigStore is lost for any reason then data will be written to
\\WAServerB\SmallStore instead.
A mechanism outside of Datacastle will be needed to make the rest of the data available – copying both
the SQL data (through backups or another mechanism) and also any data that ends up on
\\WAServerB\SmallStore. These both consist of significantly less data so the requirements on this
mechanism aren’t as high. It does need to ensure that all files on \\WAServerB\SmallStore are fully
available in the copy, making sure that a file that was half written when the copy is made gets updated
again later when it is complete.
6
Copyright © 2010 Datacastle Corporation. All rights reserved.