Mobile Testing - Bug Report
Portnov Computer School
Copyright Ivette Doss 2013
1
Golden Bug Report Rules:
• 1. One Issue at the time
• 2. Determine if it is Reproducible: try to repeat
the steps at least 3 times: type of bugs – bad
code, bad data, degrade, intermittent…
• 3. Collect logs and screenshots
• 4. Clear Title and Description
• 5. Exact Steps to Reproduce
• 6. Device and Environment Details
Copyright Ivette Doss 2013
2
The bugs that not reproducible
immediately
• Some bugs may occur once out of three
attempts. If you try to repeat it more times, and I
may only be able to repeat it five times out of
ten, or thirteen times out of twenty attempts.
• Then add the word “Intermittent” to the bug title
and include those metrics.
• That just means that the programmer will have to
try to repeat it a few times rather than once to be
able to repeat the conditions and start fixing the
bug.
Copyright Ivette Doss 2013
3
Title and Description
• Brief description, usually based around a task or
fulfilling a goal to put context around the bug.
• That helps the programmers figure out why they
should fix the bug, and what sort of priority it
should have.
• Here is an example: “While walking between WiFi connections, I tried to submit a purchase, but
the app crashed. This seems to happen when the
Wi-Fi signal is weak.”
Copyright Ivette Doss 2013
4
Exact Steps to Reproduce 1
• Write down the steps, and practice it yourself.
• Also explore to see if there are any extra steps
that you can remove and still see the problem. If
you eliminate one or two steps and it no longer
occurs, be sure to also note that in the report.
• This is useful information for diagnosing the
source of the issue. Be sure to note anything in
the environment when you are working with
steps to reproduce bugs on mobile devices.
Copyright Ivette Doss 2013
5
Exact Steps to Reproduce 2
• Remember that movement activates sensors,
different networks have better or worse
performance, transitions between network types
can cause problems, and low battery can cause
the device to behave very differently.
• You may notice a problem when you are on the
move, get back to your desk and be unable to
reproduce it. I always take screenshots with the
device (or take a picture with a different device)
Copyright Ivette Doss 2013
6
Exact Steps to Reproduce 3
• Once I work to repeat the bug, I will narrow things down:
• I had a low battery when the bug occurred. Does it also
occur when the battery is charged?
• Maybe the Wi-Fi connection wasn’t at full connection. Can I
reproduce it with full Wi-Fi?
• I noticed the bug while walking and moving the device,
which gets sensors involved. Does it still occur when I am at
my desk with the device sitting on it, without moving? In
other cases, I might have been using a cellular network.
• Can I repeat it when I’m using a more powerful connection
like Wi-Fi?
• Understanding these issues helps me create a much more
accurate bug report.
Copyright Ivette Doss 2013
7
What Should Have Happened
• Testers have great ideas on how apps can provide value to
customers and users.
• We often use these motivations as we generate test ideas. We
notice that there is a problem, but it may not be obvious to
everyone else.
• Furthermore, the programmer may only have a vague idea of a
better solution, so our ideas will help them as they solve the
problem, or as they confer with designers and other developers.
• You can also find information in specifications, requirements
documents, or in developer guidelines that are supplied with each
mobile development framework program.
• This can help provide credibility to your bug reports.
Copyright Ivette Doss 2013
8
Device and Environment Details
• At the minimum, always include the following:
• * Device manufacturer and model number (e.g. HTC One X)
• * Operating system version and sub-version (Android 4.0.3)
http://www.softwaretestingmagazine.com/videos/how-real-tests-lead-to-real-progress/
http://www.softwaretestingmagazine.com/category/videos/
http://www.softwaretestingmagazine.com/videos/bad-software-testing-good-software-testing/
Copyright Ivette Doss 2013
9
Device and Environment Details
•
•
•
You can find this information in the settings app on your device. Also be sure to
include anything about the app, including:
* The version of the software you are testing (Project Xena 1.01 Build 24) Here are
some device details to consider including when relevant:
* The date and time when you noticed the problem
* Wireless network type
* Low battery (can cause device to slow down)
* Movement and sensor interaction
* Using location services, such as GPS
* Other activity on the device: other apps running, interruptions by OS,
communication, etc.
* Apps your app depends on, such as web browsers
* Device heat (can cause device to slow down)
•
•
•
•
Here are some environmental issues to include when relevant:
* Network connection strength, or if a transition between two networks occurred
* Light conditions and weather
* User movement and travel speed
•
•
•
•
•
•
•
Copyright Ivette Doss 2013
10
Supplementary Information
• The old saying that a picture says a thousand words holds
true for bug reports.
• I may have a poor bug report, but an image or video
capture demonstrating the problem may save the day.
• What I was unable to describe well I may be able to
demonstrate perfectly. One of the most difficult pieces of
supplementary information for beginners to gather are
stack traces.
• Stack traces are complex error messages that are often
suppressed from the user interface. You see a friendly
message: “Oops, sorry, an error occurred” but in the
background, more specific information is recorded and
stored.
Copyright Ivette Doss 2013
11
Golden Bug Report Rules:
• 1. One Issue at the time
• 2. Determine if it is Reproducible: try to repeat
the steps at least 3 times: type of bugs – bad
code, bad data, degrade, intermittent…
• 3. Collect logs and screenshots
• 4. Clear Title and Description
• 5. Exact Steps to Reproduce
• 6. Device and Environment Details
• 7. Supplementary Details: Screenshots and Logs
Copyright Ivette Doss 2013
12