Best Practices 1.21.2016. - Replibit

Replibit LLC
Phone: 888 978 3610
Email: [email protected]
www.replibit.com
Replibit Best Practices
Installation/Management
Replibit Best Practices
1. Disk I/O Load- Test your Virtualization
Compression
Disk Selection
Mismatched Hard drives and broken RAID
Failed Replication Due to I/O

Affects Local Retention / Fills Appliances
2. Why We Fail Backups; Types of Backup Failures
MFT Defrag/Normal Defrag/Check Disk/SFC Scan
It is highly recommended that before performing any block level backup
installation to defragment the MFT (Master File Table) as well as a regular disk
defragmentation.
Run a Check Disk Verify and SFC Scan Verify
Turn on Boot VM Check. This is put into ConnectWise as a ticket for you. We
will either email it to you, one email per appliance with all of the images or we
will put this in as a document into ConnectWise, in a ticket.
Unique to Replibit-Non Chain Based Integrity Checks performed on every single
backup and before we log it as a successful backup:
 Backup is mounted under the hood on the appliance
 We make sure that we can traverse the directory structure
 Numerate out the directories
 Read-down the master file table and look for orphan objects
 Check for bad magic quadrants.
 Check for high fragmentation levels-which affects virtualization from
Windows
 Check for any DLL mismatches.
 Real time tracking mechanism that creates a hash file, Automatically in
the last backup the day, we run a full differential to compare every single
block in the backup set to that of the production
 Real time resilvering of data-We have coded ZFS to do real time
resilvering. So we are running a 24/7 low level, block scan or sector scan
on the disk, on the appliances in the field, and in the vault up in the sky.
 Track Window Boot DLL files- expectation of virtualization, proper
modifications to Boot DLL files- greatly impacts stability and performance
on Virtualization.
If any piece of the data set exceeds our tolerances we proactively fail the backup
job and we’ll put a ticket in ConnectWise, or email alert.
That is us telling you that you have a problem on your windows machine, that
you‘re unaware of that will affect will affect the recoverability of your data, and
your ability to virtualize the machine in a disaster.
On every single backup we perform all of those tests to log it as successful. As
an image based backup we are a true mirror.
We are a good in- good out, or garbage in-garbage out system.
How to resolve:
 Defrag, MFT defrag, check disk, SFC scan, Ultra Defrag – Full
Optimization, check on appliance for a failed disk
 If you get the error snapshot creation failed it is most likely failing VSScheck NFTS event log
 Backup Analysis tool
3. OEM & UEFI Installations, Backups and Virtualization
In many instances there is the capability to virtualize on OEM installations,
however, it is a violation of your MS licensing agreement to virtualize on OEM
BMR to dissimilar hardware / OEM installation
DELL OEM with EISA partition –FAT 32 / BCD File

https://replibit.kayako.com/Knowledgebase/Article/View/192/0/howto-fix-missing-bcd-files-with-eisa-partitions---bootvm-fails
We are a VSS aware backup we only backup NTFS VOLUMES
Many OEM installations are designed not to allow Virtualization.
Sign for UEFI Beta by emailing [email protected]
4. The Web GUI and our Virtual Switching layer is bound to ETH0.
Verify you are plugged into ETH0.
If in disaster mode when you virtualize you must be bound to ETH0 or there
will be no connectivity to the physical network.
5. Modifying ARISTOS CONFIG/Replibit Directory
Volume Exclusion
Set artistos.cfg for volume exclusions and worker threads
 http://replibit.kayako.com/Knowledgebase/Article/View/16/0/modifyin
g-the-replibit-agent---set-volumes-and-worker-thread
Modifying the Aristos Configuration File
Stop Replibit Service
Default Values on Rollout
Specifying Drive Letters
 Mitigate USB Drives and iSCSI mount points
License Key
Worker Threads
 NUM_WORKERS
 IO Loads and VSS quiesce
 Reduction of Worker threads = Reduction of Load on the Server
 Minimum Value=1 Worker Thread
AristosAgent.Exe
 Backup Engine
 Hung Events
 Self Healing/Web Update Service
 VShadow Tool
6. Windows and VSS Configuration Best Practices
VSS Settings for Backup Agent and Disk Space for *.bin Files
Make sure there is enough room for VSS calls and the creating of *.bin files or
backup will fail.
Using the “vssadmin list shadowstorage” command, view the volume shadow
copy size and make sure it is not set to “UNBOUNDED”.
The windows default installation size is set to 10% free space
We recommend the windows default value of 10%.
If there is less than 15% total free disk space on a particular volume, Replibit
recommends redirecting the shadow storage space to a different volume
For example you can redirect the storage space for the C:/ volume to D:/ by
following this command:
 vssadmin resize shadowstorage /for=c: /on=d: /maxsize=10%
Ensure all VSS writers are in a stable state by using the command:
 vssadmin list writers
Backup Time Window-Best Practice
 MS best practices recommend an increase in the backup time window of
15 minutes for every TB of disk space or LUN size.
 Note: this is not used space but is the actual volume size
Windows SBS Server Exchange VSS
 The Exchange VSS writer on Windows SBS 2003 is disabled by default
 Please ensure that you verify that the exchange VSS writer is set to
enabled.
 Refer to http://support.microsoft.com/kb/838183
Complete Uninstall of other Backup Software and Reboot
 If you have other backup software installed such as Acronis or Backup
Exec-their standard uninstaller does not completely remove the program
and leaves VSS in an unstable state. –EVENT ID 12292
 Please use their standalone uninstaller to remove program remnants and
set VSS in a stable state
 Cleaning up old VSS
 Please use their standalone uninstaller to remove program remnants and
set VSS in a stable state
 In circumstances where it is unavoidable and you MUST have a
secondary backup solution assume the other product is also VSS aware,
and ensure that both are on different backup schedules
Search for unknown large file creation and Retention
 On the windows machine you are installing the Replibit backup agent,
perform a search for files greater than 50MB’s in size, created within the
past 7 days
 SQL files and line of business applications- unexpected large files such
as SQL backups, QuickBooks backups, and PST files
 Make the destination object a machine that is not being backed up by
block level application or create scratch partition
 If large changing files are found this may create a scenario where your
appliance storage fills more quickly than expected. One example of this
scenario and steps to mitigate are as follows:
o SQL performs a scheduled task to export a backup and save
it on the server itself.
o Either change the save path to an external drive/NAS device
or shrink a partition on the server and create a new partition
solely for the SQL backup file.
o If you do the latter, you can exclude that partition from the
Replibit backup by specifying which partitions to backup.
Minimum Hardware/Software/Miscellaneous Requirements
External IP Addresses
The Vault and Management Portal are independent machines with their own
operating systems. Each requires its own external IP address.
Open Ports
LAN -Verify that ports 9090 through 10,000 are open between the server with
the Replibit Agent and the Replibit Appliance
WAN - The Appliance to Vault replication requires ports 22, 443 and 9080 be
open on the Vault side firewall. The Global Management Portal (GMP) requires
ports 22, 443 and 10,000 + the number of devices report to the GMP. For
example: 5 appliances in the field requires ports 10,000-10,004 open on the
GMP side firewall.
Appliance and Vault Installation and Requirements
It is highly recommended that the Replibit software be installed on Bare Metal
Replibit offers the freedom to choose your hardware, however, there are minimum
hardware requirements and we recommend verifying that the hardware is
compatible.
Before installation please check: http://www.ubuntu.com/certification/
Optimal Hardware Specifications:
OS: 8 GB RAM, 2 Cores
Backup Resources:
 For Every 5 Protected Systems (production servers being backed up)
 4GB RAM, 2 Cores
Additional Requirement:
 Boot VM Check Implementation 2GB RAM, 2 Cores
The appliance must have 2 hard drives (1 for OS, 1 for Storage).
The OS Hard drive needs to be at least 250GBS in size.
The Processor must support hardware assisted virtualization (Intel VT, or AMD-V).
Intel processors perform significantly better than AMD processors.
OS volume RAID-The recommended RAID configuration is to use a hardware RAID
controller for the OS hard drive and configure a RAID 1 mirror. Please check the
Ubuntu certification site to verify if your RAID controller is supported.
 http://www.ubuntu.com/certification/catalog/
Storage Volume RAID-unless you are using a SAN-it is recommended to use the built
in Software RAID within Replibit.
Replibit Software does not require the purchase of a proprietary hardware giving you
flexibility in your hardware vendor choice.
To ensure success, we highly recommend confirming that either complete
desktop/server systems and/or individual components are compatible with the
Ubuntu 12.04 / 64bit
To confirm if your complete desktop system is compatible check:
 http://www.ubuntu.com/certification/desktop/
To confirm compatibility with Ubuntu on a complete server system check:
 http://www.ubuntu.com/certification/server/
To confirm if individual components (processor, video cards, RAID controllers,
etc.) are compatible click:
 http://www.ubuntu.com/certification/catalog/
DHCP-When installing Replibit, have the hardware connected to a DHCP network.
A Static IP can be configured during the setup wizard or at any other point via the
settings tab.
Post installation and reboot, Replibit will tell you what IP address has been assigned
via DHCP. The rest of the installation will be performed via Web Browser.
In the web browser setup wizard, at the setup screen which shows your IP address,
you can click the Configure button to assign the static IP.
If you have multiple network cards select the dropdown menu and to assign the IP to
additional network cards, this is especially important if you are using a SAN and
have a storage VLAN
The Zenith BDR with a Tyan 2912 motherboard does not support Linux
BMR - Replibit’s commitment to help ensure data integrity on the boot volume,
Replibit will perform a block by block copy job in the BMR environment. We use ISCI
technology for the BMR. Please be aware that this extra data integrity check requires
a longer time to complete versus a data volume restore.
Running Replibit as a Nested VM
You must schedule a call with a Replibit Engineer prior to deploying
Replibit has a Type-1 Hypervisor embedded in it.
Replibit does not recommend being run in a virtual environment
MS Hyper-V does not support nested virtualization
VMWare ESX 5.1 functions on both the Intel and AMD platform
VMWare ESX 5.5 only works on the Intel platform
There are edits that need to be made to the Replibit Guest .vmx file. Please refer to
KB articles:
http://replibit.kayako.com/Knowledgebase/Article/View/17/0/editingvmware- settings-to-support-replibit
http://replibit.kayako.com/Knowledgebase/Article/View/20/0/vmware-ovfinstallation-guide
Licensing: Customers and Locations
The Replibit Vault is Multi-Tenant. In order to segregate the data, the concept of
customers and locations is utilized.
Prior to deploying a new customer or location, login to licensing.replibit.com with
your direct reseller username and password and define your customer and their
locations.
For Example: Customer “WidgetCo” may have two offices, a primary and a
satellite office. You would create the customer “WidgetCo” and subsequently
create two locations (Primary and Satellite) for “WidgetCo”.
When first deploying your appliance, at the “get customer screen” type in your
Reseller username and password and click the get customers button.
Replibit will download your list of unassigned customers and locations.
Assign the appropriate customer and location to the Appliance. A unique “Used ID” is
now assigned to the Appliance.
Once assigned, that customer and location combination will no longer be available
when you select the Get Customers buttons on future installations.
This prevents 2 devices from being assigned the same customer name and location.
If you wish to have more than 1 appliance at the same customer /location use our
clustering feature-please refer to the step-by-step guide page 92 to setup clusters.
KB article: http://replibit.kayako.com/Knowledgebase/Article/View/7/0/stepby-step guide
If deciding to rebuild the appliance the customer/location combination needs to be
disassociated from the licensing portal.
When deleting a Protected System on the Vault, sign into the Vault using the
customer’s credentials.
For re-installations and rebuilds using the same license key: on the Replibit
Appliance you can perform a factory reset. If the device is connected to internet it will
contact the licensing portal and flush/remove the “Used ID” automatically.
When performing a factory reset on the Appliance it will delete all data, including
configurations and settings.
To perform a factory reset, in the Replibit user interface simply click Cluster
Management then click the red “X” next to the machine listed, you will get a popup
warning this will delete all data on the device -press “OK” to confirm.
If after a factory reset, the expected Customer/Location is not available under the
“Get Customers” section of the setup wizard
To reuse same license key: You can login into the licensing portal at
licensing.replibit.com and remove your customer and location and their licenses and
then either re-add them or add a new customer and location that you are re-using
the license key for and ensure that you assign the license to the new customer and
location pair.
Glossary