CompuwareCorporation
Performance Improvements from the “Things I
Wish They’d Told Me 8 years Ago” - Visualized
Thomas A. Halinski, Systems Engineer
CMG Las Vegas
December 10, 2004
CMG Las Vegas 2004
Page 2
Performance Improvements from the
“Things I Wish They’d Told Me 8 years Ago”
- Visualized
Thomas A. Halinski
Pearls of wisdom for application performance
improvements using DB2 sound good and make a lot of
sense. But how do we know how well they work and to
what extent?
This paper will take some of the
performance tips from Bonnie Baker, a DB2 industry
expert, and measure them using STROBE. It will
explain the theory and show the actual before and after
results of implementing tips like: synchronous vs.
asynchronous I/O, index usage and stage 1 vs. stage 2
predicates.
After “seeing” the actual performance
improvements, IT personnel will be much more confident
that theory and reality can go hand in hand.
CMG Las Vegas 2004
Page 3
Birth of DB2


Once upon a time, many years ago, in the ancient
land of California, a group of developers put their
heads and hearts together and created a product
that would change the world of information
technology. On the first day, they created SQL.
On the second day, they created DB2.
— Bonnie Baker
CMG Las Vegas 2004
Page 4
Pedagogy in the DB2 World
“Blind faith in the infallibility of computer software is shortlived. Anyone who has been in this business for longer than
five minutes knows that the reliability of software is dependent
upon many things, not the least of which is the programmer's
understanding of exactly how that software works.
— Bonnie Baker, Bonnie Baker Corporation
“Thus came those who tutored. They first learned their lessons
‘the old fashioned way — they studied and applied IT’ and then
they shared.”
CMG Las Vegas 2004
Page 5
Application Programs on the
Mainframe
User
Programs
Library Routines
CPU
Teleprocessing Monitor
Database and IO Subsystems
OS/390 Operating System
Database Management Systems
I/O Subsystems
I/O Devices
I/O
Data sets
Figure 1 - A Typical Address Space
CMG Las Vegas 2004
Page 6
SQL and Your Program
APPLICATION
PROGRAM
DBAS
Database Functions
Buffering
DSNDBM1
Relational Data System
Data Manager
Buffer Manager
Stage 2 predicates
SQL statement
checking
Sorting
Optimizer
Stage 1 predicates
Indexable predicates
Locking
Various data
manipulations
Data movement
to and from DASD
Set-level Orientation
Row-level Orientation
Physical Data Access
Bufferpools
 SQL Flow – Pass 1: Program to RDS to DM to BM to PDA (VSAM Media Manager)
 SQL Flow – Pass 2: BM to DM (Stage 1 processing) to RDS (Stage 2 processing)
to the Program
CMG Las Vegas 2004
Page 7
SQL and Your Program
NOTES:
The flow of SQL from an application program goes something like this:
•It enters the RDS, which checks authorization, resolves data element names,
checks syntax, optimizes the SQL and generates an access path.
•RDS then passes the SQL to the DM.
•The DM analyzes the request for the data (table or index rows) and then calls the
BM to satisfy the request.
•If the requested data is in the buffer pools, it accesses the data and sends it back to
the DM.
•If the data is not in the buffer pools, it calls the VSAM Media Manager, which reads
the data and sends it back to the BM, which then returns it to the DM.
•On this second pass into the DM, as many predicates are applied as possible to
reduce the answer set. This is where only Stage 1 predicates are applied.
•Next, the RDS receives the data from the DM again and it applies the Stage 2
predicates, does any applicable sorting and passes the data back to the application.
•So what Bonnie Baker is telling us is that if we try and use only stage one
predicates, we will not incur the extra overhead of the Stage 2 Predicate process
done in the RDS. Let’s take a look at what our MRI of an application program shows.
CMG Las Vegas 2004
Stage 1 Predicate & CPU Consumption
Page 8
CMG Las Vegas 2004
DB2 Resource Consumption for Stage 1
Page 9
CMG Las Vegas 2004
RDS Resource Consumption for Stage 1
Page 10
CMG Las Vegas 2004
Stage 2 Predicate
Page 11
CMG Las Vegas 2004
DB2 Resource Consumption for Stage 2
Page 12
CMG Las Vegas 2004
RDS Resource Consumption for Stage 2
Page 13
CMG Las Vegas 2004
RDS Stage 2 System Services Module
Page 14
CMG Las Vegas 2004
Page 15
Stage 1 vs. Stage 2 Predicates
Resource Consumption Table
Stage 1
Stage 2
% Variance
DB2 System Services
51.86%
54.54%
5.2%
DSNXGRDS
23.03%
26.74%
16.1%
Obviously there is much more than this that contributes to the overhead
attributed to Stage 2 Predicate processing, especially since there are a
variety of Stage 2 Predicates, some BAD and some not so BAD. For a
better insight into these, see Bonnie Baker’s “Predicate Evaluation”
series in DB2 Magazine.
CMG Las Vegas 2004
Indexes – Clustered and Not
•By using indexes, access to requested rows of data in a
table is usually much faster.
•We will now look at a couple of general types of Indexes,
clustered and non-clustered.
•A Cluster ratio is an indication of the degree to which table
data in relation to an index is clustered.
•A higher cluster ratio means that the rows are ordered on
the data pages in index key sequence.
Page 16
CMG Las Vegas 2004
Indexes – Not Clustered
Page 17
CMG Las Vegas 2004
Indexes – Not Clustered…continued
Page 18
CMG Las Vegas 2004
Indexes – Not Clustered…continued
Page 19
CMG Las Vegas 2004
Indexes – Not Clustered…continued
Indexes – Clustered and Not…continued
Page 20
CMG Las Vegas 2004
Indexes – Clustered
Page 21
CMG Las Vegas 2004
Indexes – Clustered…continued
Page 22
CMG Las Vegas 2004
Indexes – Clustered…continued
Page 23
CMG Las Vegas 2004
Indexes – Clustered…continued
Indexes – Clustered and Not…continued
Page 24
CMG Las Vegas 2004
Page 25
Indexes – Clustered and Not…continued
Resource Consumption Table
Cluster Ratio 0% Cluster Ratio 100%
Ave. Execution Time
.0015
.0010
% Variance
33%
Bonnie Baker has several anecdotal papers dealing with the use of
clustered versus non-clustered indexes. Her articles “Myths of the
CLUSTERRATIO” and “ Two ways of using an index – the Normal
way and the Aunt Louise way” are examples.
CMG Las Vegas 2004
Page 26
Synchronous and Asynchronous I/O;
Singleton Select and Cursor
•Bonnie tells the story of building a patio out of bricks at her
home in Florida and uses her 2 children as deliverers of the
bricks to her as she needs them.
•Of course the bricks are dropped off in the front of the house
and she will be building the patio in the back .
• She asks her daughter, Beth, to bring her some bricks and she
goes to the front of the driveway where they are and picks one
up, walks back to where Bonnie is working and gives her one
brick and returns to what she was doing .
•Bonnie puts the brick in place and asks Beth to bring her
another one.
•Bonnie waits for Beth to return with the brick, one again.
•Bonnie and Beth were working in sync, analogous to DB2
synchronous I/O, with DB2 returning a row on a page.
CMG Las Vegas 2004
Synchronous and Asynchronous
- Singleton Select
•In the first computer program to help
illustrate this, we read a small 2000 row table
using a singleton Select. The table has no
indexes. We read the rows for the state of
Alabama, which number 99, but do this by
reading each VENDOR-ID that has a state of
Alabama.
Page 27
CMG Las Vegas 2004
Synchronous and Asynchronous I/O
- Singleton Select…continued
Page 28
CMG Las Vegas 2004
Synchronous and Asynchronous I/O
- Singleton Select…continued
Page 29
CMG Las Vegas 2004
Page 30
Synchronous and Asynchronous I/O
- Cursor
•Bonnie now continues her patio building story by bringing in her
son Scott. She asks him to get her bricks and Scott goes and
gets a wheelbarrow from the garage and fills it to its capacity,
which is, totally by chance, 32 bricks.
• He then delivers them to Bonnie and goes and gets 32 more.
•Bonnie now has these 32 bricks at her fingertips and proceeds
to put them in place, one at a time. She no longer has to wait for
more bricks to arrive and can continue her work.
•It took Scott a bit longer to bring the bricks to Bonnie than Beth,
but he would bring 32 at a time, averaging far better per brick
than the one brick method used by Beth.
•When Scott went for the second 32 bricks, he was doing
asynchronous I/O, out of synch with Bonnie.
CMG Las Vegas 2004
Page 31
Synchronous and Asynchronous I/O
- Cursor…continued
•In the next computer program to help illustrate
this concept, we read the small 2000 row table
again using a Cursor with Fetch. The table has
no indexes. We read the rows for the state of
Alabama, which number 99, but do this by using
a Cursor and having the WHERE clause with
VENDOR-STATE equal to the state of Alabama.
CMG Las Vegas 2004
Synchronous and Asynchronous I/O
- Cursor…continued
Page 32
CMG Las Vegas 2004
Synchronous and Asynchronous I/O
- Cursor…continued
Page 33
CMG Las Vegas 2004
Page 34
Synchronous and Asynchronous I/O;
Singleton Select and Cursor…continued
Run Time
SQL Ave.
Exec. Time
Singleton Select
Synchronous
3.93 sec.
Cursor/Fetch
Asynchronous
.23 sec.
% Difference
.0127 sec.
.0002 sec.
84%
94%
Bonnie Baker has written many articles dealing with DB2
performance. For further details on her “pearls of wisdom” see her
web site at “http://www.bonniebaker.com”.
CMG Las Vegas 2004
Page 35
Summary
*To Developers and Tutors, a quote for you:
“Tis God gives skill,
But not without
Men’s hands:
He could not make
Antonio Stradivari’s violins
Without Antonio.” – Stradivarius [McWilliams, P.; “DO IT! Let’s Get
Off Our Buts”, Mary Books/Prelude Press, (1991). ].
* Thank you all.
* I especially thank Bonnie Baker for her permission to use her
insights into “visualizing” what she says. Rest assured, they are of
a quantifiable value.