ObjecDves Review JUnit Review Project 1 Notes Software TesDng

10/23/15
Objec&ves Review • JUnit review • So4ware tes&ng issue: when have I tested • Describe the general tes&ng process • What is a set of test cases called? • What is unit tes(ng? • What are the benefits of unit tes&ng? • What are the characteris&cs of good unit tests? • What are the steps in a JUnit Test Case? enough? • Coverage criteria Ø How do we implement those steps? • What is test-‐driven development? Oct 23, 2015
Sprenkle - CSCI209
1
Oct 23, 2015
Sprenkle - CSCI209
2
JUnit Review Project 1 Notes • Put JUnit classes in a separate tests package • Tes&ng process: • Do not change the Car class’s API or its package 1. 
2. 
3. 
4. 
Ø Otherwise, your tests won’t work with my Car class Set up Execute code Verify output Tear down • May want to write code for Car class to help you figure out tests • Order of tests within a class? • Project 1 ques&ons? Ø Organiza&on more than coding challenge Oct 23, 2015
Sprenkle - CSCI209
3
Oct 23, 2015
Sprenkle - CSCI209
So4ware Tes&ng Issues So4ware Tes&ng Issues • How do we know if the calculator program is • How do we know if the calculator program is correct? correct? Ø How do we know that we’ve exposed all the faults? Ø How confident are we in its correctness? Ø Time? It’s been a couple hours/days/… Ø Number of test cases executed? A lot! Ø I asked my brother and he’s really smart and he says that it’s enough Ø Our customers want this product soon but we need product to be correct •  Harder to fix a4er it has been released Sprenkle - CSCI209
Ø How do we know that we’ve exposed all the faults? Ø How confident are we in its correctness? • How do we know if we’ve tested enough? • How do we know if we’ve tested enough? Oct 23, 2015
4
5
Oct 23, 2015
Sprenkle - CSCI209
6
1
10/23/15
Tes&ng Con&nuum Tes&ng Con&nuum No testing
•  Give to customer
immediately
•  Likely buggy!
•  Very little confidence in
program’s quality
Oct 23, 2015
Exhaustive
Testing
No testing
•  Test every possible input
•  Costly, impractical
•  Need to release application to
customers sometime!
Sprenkle - CSCI209
StatementCoverage
Exhaustive
Testing
• Need to execute all code • Cover (i.e., execute) all statements in the program 7
Analogy: Map coverage Oct 23, 2015
Sprenkle - CSCI209
8
Statement Coverage • Cover all statements in the program Test Suite:
num=5
Goal: Expose all the “scarecrows” public String exampleMethod(int num) {
✔ 1
String string = null;
✔ 2
if (num < 10) {
✔ 3
string = “huzzah!”;
}
// remove the leading & trailing whitespace
✔ 4
return string.trim();
}
Is this method bug-free?
Oct 23, 2015
Sprenkle - CSCI209
Program Flow 9
statement coverage but missed a branch/edge • Try covering all edges in program’s flow if( num < 10 )
true
Implicit
false Branch
Ø Also covers all nodes Ø Called Branch Coverage String string = null;
if( num < 10 )
true
string =
”huzzah!” ;
string.trim();
Oct 23, 2015
Sprenkle - CSCI209
10
exampleMethod(int num)
• Test suite had 100% String string = null;
string =
”huzzah!” ;
Sprenkle - CSCI209
What Went Wrong? exampleMethod(int num)
public String exampleMethod(int num) {
String string = null;
if (num < 10) {
string = "huzzah!";
}
return string.trim();
}
Oct 23, 2015
Implicit
false Branch
string.trim();
11
Oct 23, 2015
Sprenkle - CSCI209
12
2
10/23/15
Branch Coverage Branch Coverage exampleMethod(int num)
• Cover all branches in the program the program String string = null;
Test Suite:
num=5,
num=10
Implicit
false Branch
string==""
string
;
+”huzzah!”
condition;
String string = null;
Test Suite:
num=5,
num=10
if( num < 10 )
true
exampleMethod(int num)
• Cover all branches in if( num < 10 )
true
string.trim();
Oct 23, 2015
string.trim();
Sprenkle - CSCI209
Branch Coverage 13
the program if( num < 10 )
true
Implicit
false Branch
string =
”huzzah!” ;
}
string.trim();
Oct 23, 2015
Sprenkle - CSCI209
15
exampleMethod(int a)
Example 2 1
String str = “d”;
public String exampleMethod(int a) {
String str = "d";
2
if ( a < 7 ) {
if( a < 7 )
a *= 2;
true
false
str += "riv";
3
4
} else {
a *= 2;
str = “co”
str = “co” + str;
+ str;
str += “riv”;
}
}
if( a > 10 ) {
str += "ing";
} else {
str += "es";
}
return str.substring(6);
Oct 23, 2015
Sprenkle - CSCI209
14
public static String exampleMethod(int a) {
String str = "d";
if ( a < 7 ) {
a *= 2;
str += "riv";
} else {
str = "co" + str;
}
String string = null;
Test Suite:
num=5,
num=10
Oct 23, 2015
Example 2 exampleMethod(int num)
• Cover all branches in if( a > 10 ) {
str += "ing";
} else {
str += "es";
}
return str.substring(6);
Oct 23, 2015
Sprenkle - CSCI209
16
exampleMethod(int a)
Branch Coverage 1
String str = “d”;
2
if( a < 7 )
Test Suite:
a=3,
a=30
true
false
a *= 2;
str +=
+= "riv
“riv”;;
str = “co”
+ str;
3
str=“driv”
a=6
5
4
5
if( a > 10 )
true
6
str += “ing”;
Sprenkle - CSCI209
Implicit
false Branch
string =
”huzzah!” ;
if( a > 10 )
false
7
str += “es”;
str=“drives”
8
return str.substring(6);
17
true
6
str += “ing”;
""
Oct 23, 2015
Sprenkle - CSCI209
false
7
str += “es”;
8
return str.substring(6);
18
3
10/23/15
exampleMethod(int a)
Branch Coverage exampleMethod(int a)
Branch Coverage 1
1
String str = “d”;
String str = “d”;
2
2
if( a < 7 )
Test Suite:
a=3,
a=30
true
false
a *= 2;
str += “riv”;
str = “co”
+ str;
3
str=“cod”
a=30
4
if( a < 7 )
Test Suite:
a=3,
a=30
true
false
a *= 2;
str += “riv”;
str = “co”
+ str;
3
5
5
if( a > 10 )
true
6
if( a > 10 )
false
str=“coding” str += “ing”;
true
7
str += “es”;
19
Oct 23, 2015
exampleMethod(int a)
What Went Wrong? 2
if( a < 7 )
true
3
• 
false
Try to cover all paths in program’s a *= 2;
str += “riv”;
flow Ø Also gets all branches, nodes Ø Called Path Coverage exampleMethod(int a)
1
str = “co”
+ str;
4
flow • How many paths through this method? if( a < 7 )
false
a *= 2;
str += “riv”;
str = “co”
+ str;
if( a > 10 )
false
true
7
str += “es”;
21
Oct 23, 2015
exampleMethod(int a)
String str = “d”;
2
flow if( a < 7 )
true
false
a *= 2;
str += “riv”;
str = “co”
+ str;
3
this method? 4
5
if( a > 10 )
true
6
str += “ing”;
us path coverage? Sprenkle - CSCI209
Sprenkle - CSCI209
false
7
str += “es”;
return str.substring(6);
22
int gcd(int x, int y)
Example 3 1
• How many paths through false
7
str += “es”;
6
str += “ing”;
8
return str.substring(6);
• Cover all paths in program’s 4
5
true
Path Coverage Oct 23, 2015
true
3
8
• What test cases would give 2
if( a > 10 )
Sprenkle - CSCI209
Ø  1-‐2-‐3-‐5-‐6-‐8 Ø  1-‐2-‐3-‐5-‐7-‐8 Ø  1-‐2-‐4-‐5-‐6-‐8 Ø  1-‐2-‐4-‐5-‐7-‐8 String str = “d”;
5
6
str += “ing”;
Oct 23, 2015
20
• Cover all paths in program’s String str = “d”;
• Test suite had 100% branch return str.substring(6);
Sprenkle - CSCI209
Path Coverage 1
(and statement) coverage but missed a path 7
str += “es”;
8
return str.substring(6);
Sprenkle - CSCI209
false
6
str += “ing”;
8
""
Oct 23, 2015
4
/**
* Euclid’s algorithm to
* calculate greatest
* common divisor
*/
public int gcd( int x, int y ) {
while ( x > 0 && y > 0 ) {
if( x > y ) {
x -=y ;
} else {
y -=x;
}
}
return x+y;
}
1
while( x > 0 && y > 0 )
false
true
2
if( x > y )
true
false
3
4
x -= y;
y -= x;
5
6
8
return str.substring(6);
23
return x+y;
Oct 23, 2015
Sprenkle - CSCI209
24
4
10/23/15
int gcd(int x, int y)
Path Coverage Tes&ng Con&nuum 1
• How many paths through this method? false
while( x > 0 && y > 0 )
true
Ø Too many to count, test them all! 2
if( x > y )
true
3
1-6
1-2-3-5-1-6
1-2-4-5-1-6
1-2-3-5-1-2-3-5-1-6
1-2-4-5-1-2-4-5-1-6
1-[2-(3|4)-5-1]*-6
false
x -= y;
No testing Statement-
4
Coverage
y -= x;
Exhaustive
PathCoverage Testing
BranchCoverage
5
6
return x+y;
Oct 23, 2015
Sprenkle - CSCI209
25
Comparison of Coverage No
testing
Statement Branch
Coverage
Criterion
Advantages
Oct 23, 2015
Sprenkle - CSCI209
26
Comparison of Coverage Path
Exhaustive
Testing
Disadvantages
Statement
Coverage
Criterion
Advantages
Disadvantages
Weak, may miss
many faults
Statement
Practical
Branch
Practical, Stronger
Weaker than Path
than Statement
Path
Strongest
Branch
Path
Oct 23, 2015
Sprenkle - CSCI209
27
Oct 23, 2015
Infeasible, too many
paths to be practical
Sprenkle - CSCI209
Uses of Coverage Criteria Coverage Criteria Discussion • “Stopping” rule à sufficient tes&ng • Is it always possible for a test suite to cover all 28
the statements in a given program? Ø Avoid unnecessary, redundant tests • Measure test quality Ø No. Could be infeasible statements •  Unreachable code •  Legacy code •  Configura&on that is not on site • Do we need the test suite to cover 100% of statements/branches to believe it is adequate? Ø Dependability es&mate Ø Confidence in es&mate • Specify test cases Ø Describe addi&onal test cases needed Ø 100% coverage does not mean correct program Ø But < 100% coverage does mean tes&ng inadequacy Oct 23, 2015
Sprenkle - CSCI209
29
Oct 23, 2015
Sprenkle - CSCI209
30
5
10/23/15
Looking Ahead • Wednesday Ø Coverage tools, Design principles • Friday Ø Project 1 due • Extra Credit Opportuni&es Ø Learning on your own Oct 23, 2015
Sprenkle - CSCI209
31
6

Download Report