EXALYTICS BACKUP PROCESS
1
EPM CLARITY DIRECTOR
Name
Bernard Ash
Function
Principal
Education
Bachelor - Goucher College
17 years of professional experience, over 15 years working with Hyperion
software.
Experienc
e
• Bernard has been involved with any and all aspects of Hyperion implementations for over
15 years : Project Management, Requirements gathering, Architecture, Design,
Development, Support, Product Escalations, Training, Administration.
Was part of
Hyperion’s ATG (Advanced Technology Group) which was a special forces group of 10
hand-picked from 150.
• Customers: World Intellectual Property Organization (UN), Cigna, Chevron, VMWare, The
Hackett Group, Equinix, Fidelity, BMW, MD Anderson, Pfizer, S&P, VW, Safeway, Conoco
Philips, McDermott, Pitney Bowes, GE Capital, Alcoa, Citigroup, Columbia University, Altria
(Philip Morris), Duke Energy, Merrill Lynch, Morgan Stanley, Aspen Tech, Port Authority,
Owens & Minor, UnumProvident, El Paso Energy, Bank of the West, Verisign, ABM
Industries, NYC Housing Authority, UBS, Symantec, Advanstar, ERG, Pharmavite
Language
English, French, Spanish
Areas of
expertise
• Konvergence Shuttle, Essbase, Application Builder, MDX, Analyzer, HFR, Web Analysis
and Financial Reporting, SQR, all of Hyperion’s APIs, Hyperion Security Platform (CSS),
Smartview for Excel, Excel Essbase Add-In, Hyperion Planning, Business Rules, Hyperion
Planning Workforce & Capex
• Integration / ETL – FDM, FDMEE, DRM, EAL, HAL, SQR, ODI (Oracle Data Integrator)
2
SAVE ESSBASE! USE SHUTTLE / DODECA.
Corporate: [email protected]
www.Konvergence.com
Presenter: [email protected]
+1.800.755.4177
KONVERGENCE SHUTTLE REFERENCES
Com. Media
& Energies
Manufacture
Distribution
Life Science
Financial Services
Services,
Transport &
Constructions
SME
CONTENTS
•
•
•
•
•
•
•
•
•
•
•
•
Backup Summary
How to call the Backup
Essbase export file size
Archive the log file with the rest of the backup files
Outputting to console and log file
Essbase ASO vs. BSO exports
LOGGING
Essbase PARALLEL EXPORT
What if there are hung threads when we try to stop essbase for the backup of the bin directory?
Backup up only certain file extensions
Defrag
ESSBASE.SEC –
•
•
•
LCM Packages
•
•
•
•
Do we need to stop Essbase
Documentation from Essbase Backup and Recovery Guide
Overview including discussion on xml “migration definitions” and the Utility.sh
List of the current migration packages
Directories used
Partition Utility
DISCLAIMER
 Not everything is complete (i.e. ASO files not handled sufficiently yet) but there is a very large
amount of reusable code here. There are no guarantees, implied or explicit, about the
implementation or usage of the following code snippets. The code herein is intended as sample
code only.
6
BACKUP SUMMARY
 How to call the Backup:

BackupMaster.sh calls the Backup process and sends the output to a log.
Backup.sh
The Backup process lives in
 Essbase export file size

If Exalytics supports export files larger than 2GB AND the “EXPORTFILESIZELIMIT [appname [dbname]] n”
command
is
not
present
in
the
essbase.cfg
file
(see
https://docs.oracle.com/cd/E12825_01/epm.111/esb_techref/frameset.htm?exportfilesizelimit.htm for details on
this essbase.cfg setting), then this is NOT an issue. However, if exalytics does not support export files larger
than 2GB OR this setting is in your essbase.cfg, then you could – for large databases – create multiple export
files during an export. You need to be aware of this because, if you have to restore and there were multiple
export files created, you will have to make sure to incorporate all the export files in your restore process (and
not miss any).
 Archive the log file with the rest of the backup files

Once the log file is created, it needs to be copied into the appropriate backup folder with all the backup files
from that backup process. This should be done in the BackupMaster.sh but we need to confirm that it is
working correctly b/c it hasn’t been tested yet.
7
BACKUP SUMMARY
 Outputting to console and log file

This is not urgent b/c the process is outputting correctly to a log file but it would be nice to have it output to the
console too. The current call to output to the BackupLogs.log file is in the BackupMaster.sh. The syntax is:
./Backup.sh >> BackupLogs.log 2>&1
 Essbase ASO vs. BSO exports.

It is preferable with BSO to have a column format. This way, a load rule can be paired with a data file and
errors can be trapped when you attempt the reload. For ASO, column format exports are not supported. This
is why you will find two export functions in the script : exportDatabase and exportDatabaseASO. They each
use a different “ApplicationList”. So, it’s up to the admin to list the ASO and BSO databases separately. The
other option is to let the script loop through ALL the applications on your server. In an ideal world, we’d check
for the type of application using maxl (“display application all;” and then check the column “Storage Type”.
Based on that, the MAXL for export would either include or exclude the text “in columns”.
 LOGGING

If we want to spool output of shell script to file and also keep it routing to the command line (black) screen, call
the shell script like this : ./backupWIP.sh >> spool.log 2>&1
 Essbase PARALLEL EXPORT



If you want to enable parallel data exports, the MAXL call to export in the “ExportDatbase” function can be
adjusted to do a parallel backup. Then, on the restore side, the files can be restored using MAXL which
supports the * character for the filenames used on the import. See the essbase tech ref section about “import
database <dbs-name> data from...”
If you wish to change the number of threads created on a parallel export, you can change this variable in the
Backup.sh script : ParallelThread=1
If you wish to turn off Parallel export, you can set the ParallelThread variable to 1. TBD – this needs to be
tested to confirm that this will turn off parallel export thereby creating (assuming that exalytics can create export
files greater than 2 GB where necessary) only one export file per cube.
8
BACKUP SUMMARY
 What if there are hung threads when we try to stop essbase for the backup of the bin directory?

Code could be added after stopEssbase.sh to check for hung threads and kill them if they exist. This would
provide two benefits:
1. This would ensure that the backup of the bin directory is successful
2. Kill any existing hung threads every night. This would be a benefit b/c hung threads are not good to
have “hanging around”.
 BACKING UP THE FOLLOWING FILE EXTENSIONS

Currently, we copy all these files: otl csc rul rep eqd sel lro esm tct ind db TBD – need to add ddb and dbb?
 DEFRAG

this process does NOT defrag. If you have a global user-base, it is recommended that a defrag is NOT run
nightly b/c the aggregations would likely take too long. However, it is strongly recommended that a defrag is
run every weekend. A defrag once a week is minimum recommendation for writeback intensive applications.
 ESSBASE.SEC – DO WE NEED TO STOP ESSBASE AGENT

It is said that, between LCM, MAXL, and file system backups employing “begin archive”, we should be able to
back up everything without stopping essbase. However, a) to be on the safe side and b) because it’s
convenient to have a working copy of essbase.sec, we will make sure that .sec is backed up properly. It has
been confirmed that we do in fact have to stop the essbase agent in order to ensure a non-corrupted version of
the essbase.sec file. Yes, it is possible to get a copy of that file without shutting down the Essbase server but it
is documented and confirmed by Essbase product management that this approach has risk.
9
BACKUP SUMMARY
 “Archive.txt” - this appears in the Begin archive portion of the script. This file is not needed so it
can be ignored.
 Documentation from Essbase Backup and Recovery Guide – used this to determine important
files : http://docs.oracle.com/cd/E57185_01/EPMBK/ch04s01s02s04.html
 From this documentation, see these important pieces:
 Output of MAXL statement display application all:
10
LCM PACKAGES
 Overview including discussion on xml “migration definitions” and the Utility.sh

These “migration packages” (XML files) are created using the LCM user interface in the Shared Services admin
console. When the LCM utility is called, an xml file is passed to it so that the utility knows what to backup. The
utility file is here at /u01/app/oracle/product/EPM/Middleware/user_projects/epmsystem1/bin/Utility.sh
 The xml file contains :
1.
2.
3.
Instructions about which components to backup. In all cases except Planning, we take all possible
components. In the case of Planning, we take all but data b/c it is better to backup data using MAXL
Login information. The password is automatically encrypted the first time an XML file is used by the Utility.
The name of the backup folder that will be created. For example, from the EPMA backup package, we define
the “filePath” as: <Target type="FileSystem" filePath="/DailyBkp_EPMA_ALL"/>
 This will put all the backup files from this “migration / backup package” in a folder called
“DailyBkp_EPMA_ALL” and it will create this folder in the same directory where the xml “migration
package” resides.
11
LCM PACKAGES
 List of the current migration packages.

In each case below, first the functional name is listed, then the name of the directory we are outputting is listed:
1. CalcManager: Export Package DailyBkp_CALC_MGR_PLN_n_ESSB
2. Deployment Metadata – “DailyBkp_Deployment_Metadata”
3. EPMA – “DailyBkp_EPMA_ALL”
4. Shared Services – “DailyBkp_SS_ALL”
5. Planning
 a. APP=ASC_PLNN – Export name : “DailyBkp_PLN_ASC_PLNN”
 b. App - FOO_AH ; Export name – “DailyBkp_PLN_FOO_AH”
 All except “Essbase Data”(as pictured above for ASC_PLNN)
 c. App – FOO_CAO ; export name – “DailyBkp_PLN_FOO_CAO”
 All except “Essbase Data”(as pictured above for ASC_PLNN)
 d. FOO_CC – export – DailyBkp_PLN_FOO_CC
 All except “Essbase Data”(as pictured above for ASC_PLNN)
 e. FOO_CCn – export – DailyBkp_PLN_FOO_CCn
 All except “Essbase Data”(as pictured above for ASC_PLNN)
 g. FOO_MC – export name : DailyBkp_PLN_FOO_MC
 All except “Essbase Data”(as pictured above for ASC_PLNN)
 h. FOO_PH – export name : DailyBkp_PLN_FOO_PH
 All except “Essbase Data”(as pictured above for ASC_PLNN)
 i. FOO_PS – export name : DailyBkp_PLN_FOO_PS
 All except “Essbase Data”(as pictured above for ASC_PLNN)
 j. DMD_PLNN – export name : DailyBkp_PLN_DMD_PLNN
 All except “Essbase Data”(as pictured above for ASC_PLNN)
6. Reporting and Analysis – export name – “DailyBkp_Reporting”
12
DIRECTORIES
 LCM packages:
/u01/app/oracle/product/EPM/Middleware/user_projects/epmsystem1/import_export/


Created a place to store packages that were used in development : devbackup
Script restores into the import_export directory
 Scripts (which need to be backed up): /u01/app/oracle/product/EPM/ODIDATA/

Running backup scripts in: /u01/app/oracle/product/EPM/ODIDATA/Backups
 Essbase data files:
/u01/app/oracle/product/EPM/Middleware/user_projects/epmsystem1/EssbaseServer/essbaseser
ver1/app
13
PARTITION UTILITY
 Run PartitionLauncher.sh in the Backups directory. This passes the following parameters to the
java class:
 Set the Classpath so that the Partition class and the Java platform can find the jar files it needs to
use. This is similar to the concept of a program needing to find DLLs in a windows environment.

export
JAVA_CLASSPATH="/u01/app/oracle/product/EPM_QA/Middleware/EPMSystem11R1/common/EssbaseJavaA
PI/11.1.2.0/lib/ess_es_server.jar:/u01/app/oracle/product/EPM_QA/Middleware/EPMSystem11R1/common/Ess
baseJavaAPI/11.1.2.0/lib/ess_japi.jar:/u01/app/oracle/product/EPM_QA/Middleware/EPMSystem11R1/common
/EssbaseJavaAPI/11.1.2.0/lib/ess_maxl.jar:/u01/app/oracle/product/EPM_QA/ODIDATA/Backups/Partition.jar"
 Here, we call the java class – using the classpath we just set up above. bash.Partition is the
name of the class file. Then the parameters are in the following order :
 <user> <password> <essbase server name> <Provider services (aps) URL for the Java api> <app
name> <cube name>
 java -cp $JAVA_CLASSPATH bash.Partition UserName Password EssbaseServerName
ServerWithAPS:13080/aps/JAPI APPName CubeName
14
PARTITION UTILITY
 Calling the partition utility




To finalize the Partition backup functionality, we need to make sure it exports all the partitions per database and
puts all the partitions per database in the correct database folder. Once it looks like it is working correctly as a
standalone (calling it from PartitionLauncher.sh with just one database at a time), we can then call it from inside
the Backup.sh. I put comments in the Backup.sh which can be turned on once we are ready. Here is what the
comments look like inside the Backup.sh file:
#Here is where we can call the java class to export the partition information. An administrator login will have to
be used here
#export
JAVA_CLASSPATH="/u01/app/oracle/product/EPM_QA/Middleware/EPMSystem11R1/common/EssbaseJavaA
PI/11.1.2.0/lib/ess_es_server.jar:/u01/app/oracle/product/EPM_QA/Middleware/EPMSystem11R1/common/Ess
baseJavaAPI/11.1.2.0/lib/ess_japi.jar:/u01/app/oracle/product/EPM_QA/Middleware/EPMSystem11R1/common
/EssbaseJavaAPI/11.1.2.0/lib/ess_maxl.jar:/u01/app/oracle/product/EPM_QA/ODIDATA/Backups/Partition.jar“
#java -cp $JAVA_CLASSPATH bash.Partition user Password EsssbaseServerName
http://APSServerName :port/aps/JAPI $APPLICATION $DATABASE
#cd /u01/app/oracle/product/EPM_QA/ODIDATA/Backups
#This next line will copy the file.dat which contains the partition definitions from the Backups
directory into the appropriate app & db folder
#From there, once the backups are all done, all this stuff, including the multiple versions of file.dat
that are in their respective folders, will be copied out to D1
#TBD – you could update the class to give the partition export file a mor meaningful name.
#cp file.dat /BackupExportMigrationDefFiles/DailyBkp_Essbase/$APPLICATION/$DATABASE
15
OUTPUT
 When you point to Sample1 basic, you get the partition definition that is under the “Source
Databases” section. This indicates where the external data is going to get pushed from:
 When you point to the Sample2 Basic db, you get what’s under “source” for Sample2. Another way to say
this is that when pulling partition info from a database, you will get the external dbs that will be pushing
their data to the database you are currently querying.
16
OUTPUT
C:\JythonDEV\Essbase\lib>java -cp C:\JythonDEV\Essbase\lib\ess_es_server.jar;C:\
JythonDEV\Essbase\lib\ess_japi.jar;C:\JythonDEV\Essbase\lib\ess_maxl.jar;C:\Jyth
onDEV\Essbase\lib\Partition.jar bash.Partition admin password localhost http://l
ocalhost:13080/aps/JAPI Sample2 basic
Inside getPartInfo : Sample1.Basic
strSourceArea long appended string :
Source Area 1 : @LEVMBRS("Measures","Lev0,Measures")
Source Area 2 : @LEVMBRS("Product","SKU")
Source Area 3 : @LEVMBRS("Market","Lev0,Market")
Source Area 4 : "Actual"
strTargetArea long appended string :
Target Area 1 : @LEVMBRS("Measures","Lev0,Measures")
Target Area 2 : @LEVMBRS("Product","SKU")
Target Area 3 : @LEVMBRS("Market","Lev0,Market")
Target Area 4 : "Actual"
getPartition Begin
getPartition End
C:\JythonDEV\Essbase\lib>java -cp C:\JythonDEV\Essbase\lib\ess_es_server.jar;C:\
JythonDEV\Essbase\lib\ess_japi.jar;C:\JythonDEV\Essbase\lib\ess_maxl.jar;C:\Jyth
onDEV\Essbase\lib\Partition.jar bash.Partition admin password localhost http://l
ocalhost:13080/aps/JAPI Sample1 basic
Inside getPartInfo : Sample2.Basic
strSourceArea long appended string :
Source Area 1 : "Jan"
strTargetArea long appended string :
Target Area 1 : "Jan"
getPartition Begin
getPartition End
17
ACTUAL SCRIPT
.
18