Software Engineering Lecture 11
Software Testing
Presenter: Josef Hallberg
1
Overview
• Software testing in general
• Unit tests: What, how, why
–
–
–
–
Principles, frameworks, techniques
Benefits
Test Driven Development
Write unit tests!
• System tests
– GUI test automation story time
Why test? Tests ...
• (should) find bugs
• determine if SUT functions correctly
– Tests don't increase quality by themselves.
– Only if bugs are identified and fixed without
introducing new bugs.
• specify expected behaviour of SUT
• document anticipated use of SUT
SUT = System Under Test
Test sizes
• Small: Unit tests
– usually white box tests, i.e. tests are aware of
SUT's inner structure.
• Large: System tests
– usually black box tests, i.e. tests don't know
about SUT's inner structure.
• Medium: Integration tests
• Regression tests continuously as any
changes are made to some piece of the
infrastructure
Top down vs. Bottom up
• Top down
– Test functionality of the system as a whole
– Make stubs and mockups for emulating missing
funconalities
– Early demonstration of the main functionalities
• Bottom up
– Test as components are completed
– Need to create test-scripts
– Tests errors deep down in the dependancy
structure
5
Unit tests
• test functionality of smallest unit of
software
– a function
– a method
– grouped together: a class
•
•
•
•
are easy to automate
should be automated
(should) run fast
can give most precise error messages
System tests
• test complete system or product
– Capacity, Stress-test, Usability, Security, Performance,
Reliability, Compliance, etc.
•
•
•
•
test end user functionality
manual or automated
manual: scripted or exploratory
automated:
– e.g. shell scripts for testing servers
– e.g. capture-replay tools for GUI clients
• can be challenging to automate
Integration tests
• test collaboration of several classes,
components, subsystems, interfaces
• usually automated
• can look like large unit tests
• can look like small system tests
Unit tests
• Basic concept: Small bits of code that
–
–
–
–
–
use function under test
verify output
verify behaviour
run easily and fast
are easy to understand
• Should issue precise failure messages
Unit test frameworks
•
•
•
•
xUnit family:
SUnit, JUnit, NUnit, pyUnit,
Google C++ Testing Framework
Test::Simple, Test::More, Test::Class,
Test::Unit
• and many more
see
http://en.wikipedia.org/wiki/List_of_unit_testing_framew
orks
http://code.google.com/p/googletest/
http://www.junit.org/
#include <limits.h>
Googletest
#include "sample1.h"
#include "sample1.h"
#include <gtest/gtest.h>
// Tests factorial of negative numbers.
TEST(FactorialTest, Negative) {
EXPECT_EQ(1, Factorial(-5));
EXPECT_EQ(1, Factorial(-1));
EXPECT_TRUE(Factorial(-10) > 0);
// Returns n! (the factorial of n).
// For negative n, n! is defined to be 1.
int Factorial(int n) {
int result = 1;
for (int i = 1; i <= n; i++) {
}
// Tests factorial of 0.
TEST(FactorialTest, Zero) {
EXPECT_EQ(1, Factorial(0));
result *= i;
}
}
}
return result;
// Tests factorial of positive numbers.
TEST(FactorialTest, Positive) {
EXPECT_EQ(1, Factorial(1));
EXPECT_EQ(2, Factorial(2));
EXPECT_EQ(6, Factorial(3));
EXPECT_EQ(40320, Factorial(8));
}
Benefits: Unit tests ...
•
•
•
•
assess code quality
make code easily runnable
reduce need for debugger
can be seen as “reproducible debug
sessions”
• or as “extended compile time checks”
• create confidence in your code
Benefits: Unit tests ...
•
•
•
•
•
can serve as specification
can serve as documentation
are a precondition for refactoring
improve design
increase productivity
Test Driven Development (TDD)
• write failing test for bug or new feature
• write enough code to get new test
compiling
• run all tests and see new test failing
• write enough code to make new test pass
• run all tests and see them pass
• refactor
• start over
TDD benefits
• Ensures that tests get written
• Spec is written before code
• Think about interface usage before interface
design
• Design for Testability from the start
• Code can be run as soon as it's finished
– no more dreading upcoming debugger sessions
see
http://googletesting.blogspot.com/2008/09/test-first-is-fun_08.html
Capture Replay Tools
• Common way to automate: “GUI level
macro recorders”
• Scripting language – often Basic or Custom
– GetEdit(), SetEdit(), ClickButton() etc.
• Often give quick and easy results
• Common problems:
– Test maintenance
– Test flakiness
GUI Test Automation as
Feedback Control Problem
• Regard GUI application as a machine to
control
• Get sensors in place
• Closely observe application for deviant
behaviour
– Provide corrective measure where reasonable
– Fail test case otherwise
Movies of GUI testing
Test of an iPhone app using JUnit
And now…
SOME LIVE DEMOS
18
Questions?
Feel free to contact me:
Josef Hallberg
Mail: [email protected]
Phone: 0920 49 31 77
19