Track the data manipulation language operations
in your DB2 database
Protect against data loss using DB2 Recovery Expert
Skill Level: Intermediate
Amol D. Barsagade ([email protected])
Sr.Database consultant/IT Specialist
IBM India Software Lab
22 May 2008
Protecting data against any failure is a huge challenge for the DBA at any
organization. In production and 24x7 environments where databases are
mission-critical, any data loss is unacceptable. To successfully perform point-in-time
recovery, you must have sufficient information about all the Data Manipulation
Language (DML) operations that have taken place. This article shows you how to
keep track of all DML operations that occur against a database, even down to a
specific table. Using the IBM® DB2® Recovery Expert (RE) tool and the db2la
command, you can track DML operations along with the time that they occurred.
Introduction
The DB2 Recovery Expert (DB2 RE) is a tool that you can use to analyze the DB2
transaction log files and generate reports detailing changes to tables or table spaces
over specific time durations. These reports identify who has made changes, and
thus are useful for certain types of auditing and compliance requirements. In
addition, DB2 RE can undo or redo any of the changes.
In order to make use of the db2la command for log analysis, you must first install the
DB2 RE tool. The following sections illustrate the use of the db2la command, and
show you how to perform these functions using the graphical interface as well.
The scenario described in this article uses the following system and database
Track the data manipulation language operations in your DB2 database
© Copyright IBM Corporation 1994, 2007. All rights reserved.
Page 1 of 23
developerWorks®
ibm.com/developerWorks
configuration:
Table 1. System and Database configuration(s)
Component
Description
Operating system
Windows® XP Service Pack 2 / RHEL 4 / RHEL
5
IBM DB2 Product
DB2 9.5 Enterprise Server Edition (ESE)
Database name
TESTDB1
Table space name
TS1
Table name
Tab1
Containers in TS1
c1.dat, c2.dat, c3.dat, c4.dat
Note: In order to obtain a detailed DML report for database objects including tables,
table space, and so on, make sure the logging mode is set to archival logging.
How to change the logging mode
When a new DB2 database is created, circular logging mode is the default. In order
to change the mode from circular to archival, perform the following steps:
1.
Set the archive destination
a.
Create a folder on disk (for example, e:\db_name\archive) where
there is sufficient disk space to store archived log files. As a best
practice, keep your archived log file destination separate from the
active log file destination.
b.
To set the archival destination, create a directory called "archive"
on a disk (for example, e:\testdb1\archive) where you have
sufficient disk space so that inactive log files will be archived to this
location once the active log directory is filled.
2.
From a DB2 command window, enter the TERMINATE command to
terminate the connection to the database.
3.
From a DB2 command window, enter UPDATE DB CFG FOR db_name
USING LOGARCHMETH1 "Disk:e:\db_name\archive" to update the
archived log file destination. (Specifying a path for the archived log has
the effect of turning the Archival logging mode on).
4.
Enter the CONNECT TO db_name command to reconnect to the
database.
Track the data manipulation language operations in your DB2 database
Page 2 of 23
© Copyright IBM Corporation 1994, 2007. All rights reserved.
ibm.com/developerWorks
5.
developerWorks®
After connecting, you will be get following warning:
SQL1116N A connection to or activation of database <db_name> cannot
be made because of backup pending: SQLSTATE=57019
This occurs because the database logging mode was changed from
circular to archival mode, and a full database backup must be taken
because a backup taken when the database was in circular logging is not
sufficient when switching logging modes.
6.
Take a full database backup using a command such as the following:
BACKUP DATABASE db_name TO d:\db_name\backup
7.
Use the CONNECT TO db_name command to connect to the database
again. This time you should be successful.
Method 1. Using the db2la command line tool
Some of the db2la command options are shown in Table 2. The following examples
show how to use these options.
Table 2. db2la command options
Component
Description
-d
Name of the database
-b
Start date and time for log analysis
-e
Stop date and time for log analysis
-n
Table name
-t
Table space name
-D
Generates detailed report
-f
Output file for generated undo SQL
-q
Specifies SQL generation should be performed
while detailed report is being generated
-r
Generates redo SQL(can only be used with -q
option)
-u
Generates undo SQL(can only be used with -q
option)
Step 1. First identify the table space of the table you wish to investigate: In this
example, we query the system catalog for the table space of table TAB1.
Track the data manipulation language operations in your DB2 database
© Copyright IBM Corporation 1994, 2007. All rights reserved.
Page 3 of 23
developerWorks®
ibm.com/developerWorks
SELECT tabname,tbspace FROM syscat.tables WHERE tabname='TAB1'
The results come back and show that this table resides in the TS1 table space
Step 2. Using the table and table space name, you can generate a report of the DML
operations that have occurred against the table.
In this example, the database is called TESTDB1, the table is called TAB1, and the
table space is called TS1.
db2la -d testdb1 -n tab1 -t ts1
Note: If your database is in circular mode, you may receive the following error
message:
ARY0109E Database is not properly configured.
To use the DB2 RE product, roll-forward recovery must be enabled for the database.
Step 3. By specifying the redo option and providing a begin and end time, db2la can
produce a report that contains the SQL statements necessary to redo SQL
operations against the specified table during the identified duration:
db2la
-d testdb1 -b "2007/07/30" -e "2008/01/08" -n tab1 -t ts1 -D -q r
Step 4. Similarly, specify the undo option to generate the SQL statements in the
report that would undo any operations against the table within the identified time
duration:
• db2la -d testdb1 -b "2007/07/30" -e "2008/01/08" -n tab1 -t ts1 -D -q u
Step 5. Finally, you can output the reports to a file, along with redo and undo SQL
for the specified table during the identified time duration:
• db2la -d testdb1 -b "2007/07/30:13:21:43" -e "2008/01/08:19:40:00" -n
tab1 -t ts1 -D -q r -q u -f "e:\testdb1\report\tab1"
Method 2. Using the Graphical User Interface (GUI)
To generate reports using Log Analysis, perform the following steps.
Track the data manipulation language operations in your DB2 database
Page 4 of 23
© Copyright IBM Corporation 1994, 2007. All rights reserved.
ibm.com/developerWorks
developerWorks®
Step 1. Launch DB2 RE and connect to the database as Figure 1 shows. The main
DB2 RE window then appears.
Figure 1. Database connection through DB2 RE
Step 2. In the main menu, select Tools -> Log Analysis as Figure 2 shows.
Figure 2. Accessing log analysis
Step 3. Click the Report tab and choose the options as per the requirements.
From the Report tab, you have the following options:
• Specify report options
Figure 3. Log analysis report
Track the data manipulation language operations in your DB2 database
© Copyright IBM Corporation 1994, 2007. All rights reserved.
Page 5 of 23
developerWorks®
ibm.com/developerWorks
Selecting this option runs the log analysis report based on filters, such as table
name and table space name that are specified in the window.
• Run complete report.
Selecting this option runs a complete report without any filters for all objects starting
from transaction log number 0. You can control the starting log number by
configuring the startlog parameter. With this parameter, the log analysis process
starts from the most recently archived log file.
Step 4. Click on the Log File Options tab as Figure 4 shows.
• Choose the All Logs option unless you have a specific reason for
specifying only a subset of logs.
• Click OK.
Figure 4. Log details
Track the data manipulation language operations in your DB2 database
Page 6 of 23
© Copyright IBM Corporation 1994, 2007. All rights reserved.
ibm.com/developerWorks
developerWorks®
The DML operations for the specific table and table space you specified will be
displayed as Figure 5 shows.
Figure 5. Log analysis report for TAB1 table
Step 5. Double-click on the table entry to see more details as Figure 6 depicts.
Figure 6.Log analysis detailed report for TAB1 table
Track the data manipulation language operations in your DB2 database
© Copyright IBM Corporation 1994, 2007. All rights reserved.
Page 7 of 23
developerWorks®
ibm.com/developerWorks
The DML entries, along with their timestamp, are provided which is required for
point-in-time recovery.
Step 6. To take analysis one step further, you can check for any deleted rows.
• Double-click the Table listed in step 5 and follow the next steps.
Step 7. Select the Generate transaction detailed report only option to generate
the report shown below in Figure 8.
Figure 7. Generating transaction details
Track the data manipulation language operations in your DB2 database
Page 8 of 23
© Copyright IBM Corporation 1994, 2007. All rights reserved.
ibm.com/developerWorks
developerWorks®
Figure 8. Generating transaction details for TAB1 table
By following the above steps, either by command line or GUI, you can easily monitor
the DML statements that have been run against your database and make use of that
information for point-in-time recovery.
Track the data manipulation language operations in your DB2 database
© Copyright IBM Corporation 1994, 2007. All rights reserved.
Page 9 of 23
developerWorks®
ibm.com/developerWorks
Recovery scenarios
To understand and perform the point in time recovery with DB2 RE, you must first
understand the concept of a versioning repository. The versioning repository stores
definition information about database objects so that you can recover these object if
they are dropped accidentally. The versioning repository does not store any data for
the tables; it only stores the DDL structure of the table.
You have to explicitly create and update a versioning repository by using one of the
following methods:
GUI
1.
Start DB2 RE.
2.
Connect to the target database.
3.
From the menu bar, select Tools > Versioning Repository > Create
4.
Click Run to execute the DDL.
Windows command line
1.
Use the db2rewin32 command, for example:
• db2rewin32 -is:javaconsole createversionrepository -d
database_name -u user_name -p password
Linux/UNIX command line
1.
Log on as the instance owner to run the db2recex command, for
example:
• db2recex createversionrepository -d database -u username -p
password
Scenario 1. Recovering the dropped table to a point-in-time
using db2re
Table 3 shows the system and database configurations.
Table 3. System and database configuration(s)
Track the data manipulation language operations in your DB2 database
Page 10 of 23
© Copyright IBM Corporation 1994, 2007. All rights reserved.
ibm.com/developerWorks
developerWorks®
Component
Description
Operating system
Windows XP Service Pack 2 / RHEL 4 / RHEL 5
IBM DB2 product
DB2 9.1 Fix Pack 4 / DB2 9.5 Enterprise Server
Edition(ESE) Fix Pack 0
Database name
TESTDB1
Table space name
TS1
Table name
Tab1
Tool
IBM DB2 Recovery Expert Version 2.1
Step 1. Sever all connections to the database
TERMINATE
FORCE APPLICATION ALL
Step 2. Launch DB2 RE and update the versioning repository shown in Figure 9
• Select the database name
• Go to Tools > Versioning Repository > Update
• Click Run to update the versioning repository
• Exit DB2 RE
Figure 9. Database status
Track the data manipulation language operations in your DB2 database
© Copyright IBM Corporation 1994, 2007. All rights reserved.
Page 11 of 23
developerWorks®
ibm.com/developerWorks
Step 3. Perform a full database backup
FORCE APPLICATION ALL
BACKUP DATABASE testdb1 TO c:\testdb1\backup
Note down the timestamp the backup image receives, for
example,"20080109143109". This timestamp will be required if you restore it.
Step 4. Simulate a table failure
For this example, assume that a table called TAB1 has 10 records . Perform some
operations on the table that will generate log records, and then drop the table (to
simulate accidental deletion). Listing 1 shows code that will simulate accidental
deletion.
Listing 1. Simulated table failure
CONNECT TO testdb1
INSERT INTO tab1 VALUES(11)
INSERT INTO tab1 VALUES(12)
INSERT INTO tab1 VALUES(13)
COMMIT
TERMINATE
ARCHIVE LOG FOR DATABASE testdb1
CONNECT TO testdb1
INSERT INTO tab1 VALUES(14)
Track the data manipulation language operations in your DB2 database
Page 12 of 23
© Copyright IBM Corporation 1994, 2007. All rights reserved.
ibm.com/developerWorks
developerWorks®
INSERT INTO tab1 VALUES(15)
DELETE FROM tab1 WHERE NO=14
DELETE FROM tab1 WHERE NO=15
COMMIT
SELECT * FROM tab1 /* check 13 committed values from table Tab1*/
TERMINATE
ARCHIVE LOG FOR DATABASE testdb1
CONNECT TO testdb1
DROP TABLE tab1
SELECT * FROM tab1
The system returns the following error message:
Error: SQL0204N "Administrator.TAB1" is an undefined name.
Step 5. Launch DB2 RE and connect to the testdb1 database.
• Click Next
Step 6. Drop the table selection.
• From the Object Selection, select the Table radio button and click the
browse button shown below in Figure 10.
Figure 10. Dropped table Selection
Step 7. Retrieve the dropped table information.
From the Select a table to recover tab dialog, click Retrieve to get the dropped
table information which displays in the color red as Figure 11 shows. You can also
specify the dropped table's owner and name.
Click Next.
Select the dropped table and click OK
Figure 11. Select a table to recover dialog
Step 8. Select the point in time to recover.
To find out the exact timestamp at which the table was dropped, use the LIST
Track the data manipulation language operations in your DB2 database
© Copyright IBM Corporation 1994, 2007. All rights reserved.
Page 13 of 23
developerWorks®
ibm.com/developerWorks
HISTORY command as follows.
LIST HISTORY DROPPED TABLE ALL FOR DATABASE testdb1
You can easily determine the timestamp at which the table was dropped (for this
example, assume it is 20080109150208); however, you cannot use this exact
timestamp to recover the committed values, so you should use the timestamp value
just before it (in this example, 20080109150200 as shown in Figure 12 below).
Figure 12. Point-in-time selection
Step 9. Specify the timestamp and choose the latest backup image.
Specify the timestamp as 20080109150200 and click the Advanced tab. Choose the
latest backup image from the list shown in Figure 13 and click OK.
Figure 13. Advance point-in-time selection with backup image
Track the data manipulation language operations in your DB2 database
Page 14 of 23
© Copyright IBM Corporation 1994, 2007. All rights reserved.
ibm.com/developerWorks
developerWorks®
Step 10. Specify the desired recovery options.
To perform the desired recovery operation, choose the appropriate target table
space from the drop-down list.
• Click Next to get the summary.
Step 11. Summary
Review the actions and click Run Now as Figure 14 shows. Click OK to proceed
with the operation. After clicking OK, the system begins recovering the dropped
object. Once the process is finished, a summary shows the success of the operation,
as well as the number of records recovered.
Figure 14. Summary
Track the data manipulation language operations in your DB2 database
© Copyright IBM Corporation 1994, 2007. All rights reserved.
Page 15 of 23
developerWorks®
ibm.com/developerWorks
Step 12. Verify the recovered table.
From the command line, issue the following commands:
CONNECT TO testdb1
SELECT * FROM tab1
Verify the recovered data.
Scenario 2. Table space point-in-time recovery using db2re
Table 4. System and database configuration(s)
Component
Description
Operating system
Windows XP Service Pack 2 / RHEL 4 / RHEL 5
IBM DB2 product
DB2 9.1 Fix Pack 4 / DB2 9.5 Enterprise Server
Edition (ESE), Fix Pack 0
Database name
TESTDB1
Table space name
TS2
Tool
IBM DB2 Recovery Expert Version 2.1
If a table space gets dropped or corrupted, the tables and the data inside that table
space is not accessible. To recover such table spaces, you must have full database
backup, and the database must be configured to do archival logging.
Step 1. Sever all database connections.
From the command line, issue the followings commands:
TERMINATE
Track the data manipulation language operations in your DB2 database
Page 16 of 23
© Copyright IBM Corporation 1994, 2007. All rights reserved.
ibm.com/developerWorks
developerWorks®
FORCE APPLICATION ALL
Step 2. Launch DB2 RE and update the versioning repository as Figure 15 shows.
• Open DB2 RE
• Select the database name
• Go to Tools > Versioning repository > Update
• Click Run to update the versioning repository
• Exit from DB2 RE
Figure 15. Database status
Step 3. Take a full offline database backup
From the command line, issue the following command:
• BACKUP DATABASE testdb1 TO c:\testdb1\backup
Step 4. Simulate the table space failure
Track the data manipulation language operations in your DB2 database
© Copyright IBM Corporation 1994, 2007. All rights reserved.
Page 17 of 23
developerWorks®
ibm.com/developerWorks
Simulate the failure, and drop the table space (explicitly). Issue the following
commands:
CONNECT TO testdb1
DROP TABLESPACE ts2
TERMINATE
FORCE APPLICATION ALL
Step 5. Launch DB2 RE and connect to the database
• Click Next
Step 6. Selection of dropped table space ts2
From the object selection tab:
• Select the table space radio button
• Click the browse button to launch a new window asking you to
"Select a table space to recover."
• Click Retrieve to display the dropped table space in a red color as
Figure 16 shows.
• Select the dropped table space
• Click OK
Figure 16. Select a table space to recover
Track the data manipulation language operations in your DB2 database
Page 18 of 23
© Copyright IBM Corporation 1994, 2007. All rights reserved.
ibm.com/developerWorks
developerWorks®
Step 7. Determine the exact timestamp for the dropped table space
To find out the timestamp at which the table space was dropped, use the LIST
HISTORY command.
LIST HISTORY ALL FOR DATABASE testdb1
You can see the exact dropped table space timestamp 20080108152909; however,
this timestamp cannot be used because a timestamp value just before it is needed in
order to recover up to the last committed value. In this example, we use
20080108152900 as the timestamp for recovery.
Step 8. Specify the timestamp
Apply the timestamp as 20080108152900 as Figure 17 shows.
Figure 17. Point-in-time selection
Track the data manipulation language operations in your DB2 database
© Copyright IBM Corporation 1994, 2007. All rights reserved.
Page 19 of 23
developerWorks®
ibm.com/developerWorks
• Click Advanced
• Select the Backup Images radio button as shown in Figure 18 below.
• Select the latest Backup Image
• Click OK
• Click Next to select the Recovery Path
Figure 18.Advance point-in-time selection with backup image
Track the data manipulation language operations in your DB2 database
Page 20 of 23
© Copyright IBM Corporation 1994, 2007. All rights reserved.
ibm.com/developerWorks
developerWorks®
Step 9. Confirm the recovery path.
From the Recovery Path screen, select Restore from the versioning repository as
Figure 19 shows.
Figure 19. Selecting recovery path
• Click the browse button to confirm the dropped table space "Ts2"
• Click Next
• The system will display the Recovery Options screen
• Click Next
• System will display the summary for the requested recovery.
Step 10. Summary
• Click Run Now
• Click OK once you are prepared to begin the process shown in Figure
20.
Once you click OK, the system starts recovering the dropped table space. It will
recreate and recover the dropped table space from the versioning repository.
Figure 20. Summary
Track the data manipulation language operations in your DB2 database
© Copyright IBM Corporation 1994, 2007. All rights reserved.
Page 21 of 23
developerWorks®
ibm.com/developerWorks
Step 11. Verify the recovered table space.
From the command line, issue the following commands:
CONNECT TO testdb1
LIST TABLESPACES SHOW DETAIL
Conclusion
This article described the importance of the log analysis function available with DB2
Recovery Expert (DB2 RE). To track the timestamp associated with a DML
statement, you can make use of the log analysis functionality and can perform
critical point-in-time recoveries.
Track the data manipulation language operations in your DB2 database
Page 22 of 23
© Copyright IBM Corporation 1994, 2007. All rights reserved.
ibm.com/developerWorks
developerWorks®
Resources
Learn
• "DB2 for z/OS and OS 390 Data management Tools Update" (developerWorks,
Feb 2002) provides an explanation for DB2 z/OS
• "Using DB2 Incremental Backup" (developerWorks, May 2002) describes the
how incremental backup in DB2 works, when you might want to use it, and
strategies for ensuring smooth recoveries.
• "IBM DB2 Recovery Expert for Multiplatform User's guide"
• "IBM Information management various tools "
• "DB2 recovery Expert V1.1 Installation information on multiplatform "
(developerWorks, Nov 2004) describes Installation and configuration
information on various platform.
Get products and technologies
• Download DB2 9.5 Trial version, a no-charge version of DB2 9.5 Express
Edition for the community that offers the same core data features as DB2
Express Edition and provides a solid base to build and deploy applications.
Discuss
• Participate in the discussion forum for this content.
• Join the discussion on How to Track DML for specific Table in DB2
• Check out developerWorks blogs and get involved in the developerWorks
community.
• Participate in the discussion forum.
About the author
Amol D. Barsagade
Amol D. Barsagade is a Senior Database Consultant/IT Specialist with the IBM
Business Partner Technical Enablement team and based at the India Software Lab.
Amol works with Business Partners and specializes in providing mission critical DB2
solutions for database recovery, performance tuning, migration, data warehousing
and application development. Amol has a bachelor's degree in computer science and
has several years of experience working with other relational database systems
including Oracle and SQL Server.
Track the data manipulation language operations in your DB2 database
© Copyright IBM Corporation 1994, 2007. All rights reserved.
Page 23 of 23