The Burden of Proof
Jonathan Lewis
www.jlcomp.demon.co.uk
Agenda
Who am I ?
Why do we need proof ?
What is proof ?
Constructing proofs
Where next ?
Q&A
NoCOUG Keynote
The Burden of Proof
2Jonathan Lewis
© 2002-2003
Who am I ?
Independent Consultant.
18 years experience.
Design, Strategy, Reviews,
Briefings, Seminars,
Tutorials, Trouble-shooting
www.jlcomp.demon.co.uk
NoCOUG Keynote
The Burden of Proof
3Jonathan Lewis
© 2002-2003
Why do we need proof ?
How does an architect design a house ?
–
–
–
–
Soil Characteristics
Material Strengths
Heating requirements
Illumination
Industry Standard Formulae
Why are so many Cathedrals standing after 900 years ?
Because they are the second, third or fourth attempt.
NoCOUG Keynote
The Burden of Proof
4Jonathan Lewis
© 2002-2003
"Proof" by intuition
C = pD
C = 2pr
p = 3.1415926 ….. @ 3
C @ 6r
To add 36 inches to the circumference,
you have to add 6 inches to the radius.
NoCOUG Keynote
The Burden of Proof
6Jonathan Lewis
© 2002-2003
"Proof" by intuition
The manual says:
"redo log space requests
should be close to zero".
Problem:
One transaction takes 0.1 seconds, which is good
The overnight batch takes 3 hours, which is bad.
The "batch" emulated 100,000 transactions !
100,000 * 0.1 = 10,000 seconds.
10,000 / 3,600 = 2 hours 47 minutes
You have to 'fix' the transaction.
NoCOUG Keynote
The Burden of Proof
7Jonathan Lewis
© 2002-2003
Skinner Boxes
Which pig does
all the work ?
NoCOUG Keynote
The Burden of Proof
8Jonathan Lewis
© 2002-2003
"Proof" by anthropomorphism
• Pigs are not people.
• The Oracle database engine is not a person.
How would you handle this query:
select max(partitioning_col)
from
partitioned_table;
NoCOUG Keynote
The Burden of Proof
9Jonathan Lewis
© 2002-2003
"Proof" by anthropomorphism
Quote from Metalink forum:
I would have thought Oracle was smart
enough to know that it just has to go to
the top partition to get max(pt_col)."
What if the top partition is empty ?
Smart code for max() in indexes only appeared in 8.1.
One day partition code might start from the top down.
How will you discover that this new feature has appeared.
NoCOUG Keynote
The Burden of Proof
10Jonathan Lewis
© 2002-2003
"Proof" by self-deception
The truth is out
there somewhere
NoCOUG Keynote
The Burden of Proof
11Jonathan Lewis
© 2002-2003
"Proof" by self-deception
select count(*) from t1;
select count(1) from t1;
select count(pk_column) from t1;
SORT (AGGREGATE)
BITMAP CONVERSION (COUNT)
BITMAP INDEX (FAST FULL SCAN) OF T1_BIT
NoCOUG Keynote
The Burden of Proof
12Jonathan Lewis
© 2002-2003
The Dodo argument
NoCOUG Keynote
The Burden of Proof
13Jonathan Lewis
© 2002-2003
An Oracle Dodo
I had I/O problems and saw lots of sorts
(disk) in v$sysstat. My I/O problems went
away when I increased my sort_area_size.
The Dodo says:
If you have I/O problems and lots of sorts to
disk, increase your sort_area_size.
The scientist says:
What I/O stopped happening, and why ?
NoCOUG Keynote
The Burden of Proof
14Jonathan Lewis
© 2002-2003
Proving or Testing
Why do we need proofs ?
Why do we test ?
What's the difference ?
We test for our own benefit.
A proof is what we pass on to others.
The difference is in the publication.
NoCOUG Keynote
The Burden of Proof
15Jonathan Lewis
© 2002-2003
Testing.
Why do we test ?
It avoids expensive mistakes.
How do we test ?
As cost-effectively as possible.
NoCOUG Keynote
The Burden of Proof
16Jonathan Lewis
© 2002-2003
Why do we test ?
Here be dragons
To learn anything other than the stuff you find in books, you need to be
able to experiment, to make mistakes, to accept feedback and to try again.
Charles Handy
The Manuals
NoCOUG Keynote
The Burden of Proof
17Jonathan Lewis
© 2002-2003
Case Study (1)
create table t1 (
id
number
primary key,
v1
varchar2(20)
)
organization index
partition by range (id)
subpartition by hash (id) (
partition p1 values less than (10)
);
ORA-25198: only range partitioning ...
NoCOUG Keynote
The Burden of Proof
18Jonathan Lewis
© 2002-2003
How do we test ?
• Don't even try to be realistic to start with test the concept, not the detailed target.
• If you want to find out if 'feature X' works:
imagine the simplest possible case where it
is most likely to happen
• If you need 'feature X' to work: don't design
a test to see if it works - imagine the
simplest possible case that breaks it.
NoCOUG Keynote
The Burden of Proof
19Jonathan Lewis
© 2002-2003
Designing tests is hard
Try drawing a 'random' triangle
NoCOUG Keynote
The Burden of Proof
20Jonathan Lewis
© 2002-2003
Case study (2)
create view daily_sales_pv as
select s.* from t_1998_01_01 s, dates d
where s.day_code = d.day_code
union all
...
union all
select s.* from t_1998_12_13 s, dates d
where s.day_code = d.day_code
;
The designer thought this was a partition view, and tested
it with 1,000 items of data. Response time was good.
NoCOUG Keynote
The Burden of Proof
21Jonathan Lewis
© 2002-2003
How do we test ?
• Know how to measure the results
–
–
–
–
–
sql_trace
tkprof
netstat
iostat
v$ snapshots
Not the clock !
• Know how to generate data cheaply
– dbms_random
– trunc(), mod(), sequences
NoCOUG Keynote
The Burden of Proof
22Jonathan Lewis
© 2002-2003
Proving or Testing
A Test becomes a Proof when:
it has a specification
the results are reproducible
alternative explanations have been eliminated
it is published
it survives peer-group review
NoCOUG Keynote
The Burden of Proof
23Jonathan Lewis
© 2002-2003
Example of test/proof
Is memory 14,000 times faster than disk ?
Assume disks can do 100 I/Os per sec
So the hypothesis claims that you could
do 1.4M logical I/Os per sec. in memory
NoCOUG Keynote
The Burden of Proof
24Jonathan Lewis
© 2002-2003
System used to test
O/S
CPU Speed
Oracle
db_block_size
Windows 2000
700 MHz
8.1.7.4
8K
create table t1 (n, v, primary key(n))
organization index as
select rownum n, to_char(rownum) v
from all_objects
where rownum <= 21;
NoCOUG Keynote
The Burden of Proof
25Jonathan Lewis
© 2002-2003
Peer Group review
declare
What's wrong
m_v varchar2(32);
with the test ?
begin
for i in 1..140000 loop
select v into m_v from t1 where n = 1;
end loop;
end;
Consistent Gets = 140,000
The hypothesis predicts 0.01 CPU seconds
Actual CPU used by this session = 16.18 seconds
NoCOUG Keynote
The Burden of Proof
26Jonathan Lewis
© 2002-2003
Second attempt
select count(*) X from t1
connect by n > prior n
start with n = 1;
CR gets
1,000,000
Buffer is pinned
Total
500,000
1,500,000
The hypothesis predicts a little over 1 CPU second
Actual CPU used by this session:
NoCOUG Keynote
The Burden of Proof
15.08 seconds
27Jonathan Lewis
© 2002-2003
Warning:
Is memory 14,000 times faster than disk ? (No)
Does it matter ?
Was it worth the effort of proving the point ?
Is it a generally applicable observation ?
Be careful about the line between pure
research and practical investigation.
NoCOUG Keynote
The Burden of Proof
28Jonathan Lewis
© 2002-2003
Relevance
declare
m_v varchar2(32);
begin
for i in 1..140000 loop
select v into m_v from t1 where n = 1;
end loop;
end;
NoCOUG Keynote
Does a small table need an index ?
Should a small table BE an index (IOT) ?
Or a single table hash cluster ?
The Burden of Proof
29Jonathan Lewis
© 2002-2003
Quality
Designing detailed tests is hard work.
Covering all the options is difficult.
The best results come from group efforts.
Thorough proofs tend to be cumulative
– but technology changes push proofs out of date
NoCOUG Keynote
The Burden of Proof
30Jonathan Lewis
© 2002-2003
Where next ?
Always ask for some proof
– Books and magazines
– FAQs
– User Group websites
Questions to ask of authors:
– How would you demonstrate that ?
– Under what circumstances is this true ?
– Do you have a sample script ?
NoCOUG Keynote
The Burden of Proof
31Jonathan Lewis
© 2002-2003
Where next ?
Document your own tests
Publish your findings - especially problems.
– www.jlcomp.demon.co.uk/faq/ind_faq.html
– www.dbazine.com
Lobby Oracle Corp. for realistic examples
– They are getting much better
– There are still too many single table examples
NoCOUG Keynote
The Burden of Proof
32Jonathan Lewis
© 2002-2003
The Benefits
•
•
•
•
You are not the first !
You can build on previous examples
It enhances your skills and understanding
You avoid problems earlier and earlier
NoCOUG Keynote
The Burden of Proof
33Jonathan Lewis
© 2002-2003
Final Thoughts
When you have eliminated the impossible, whatever remains,
however improbable, must be the truth.
Sir Arthur Conan Doyle
If I have seen further than others, it is because I was
standing on the shoulders of the giants who went before.
Sir Isaac Newton
Please provide a reproducible test case.
Oracle Support
NoCOUG Keynote
The Burden of Proof
34Jonathan Lewis
© 2002-2003