How to Read Research Papers?
Xiao Qin
Department of Computer Science and
Software Engineering
Auburn University
http://www.eng.auburn.edu/~xqin
[email protected]
Some slides are adapted from notes by Nitin H. Vaidya at the University of Illinois at
Urbana-Champaign
Why read papers?
• To know what’s happening
• Avoid reinventing the wheel
– does happen commonly,
too many wheels already
• Find interesting research topics
Why not to read papers?
• Cannot read everything
• Should not read everything
• Can suppress innovation
– once you see solutions using a
particular theme, often hard to
think differently
Read or not to read,
that is the question
• Read, of course
• Know what’s important
• Know what can be ignored without
significant loss of information
Where to search?
• When I was a Master’s student…
• When I was a Ph.D. student…
• Now…
Where to search papers?
• IEEEXplore
• ACM Digital Library
Where to search papers?
• Google Scholar
• Personal Web Pages
What to read?
• Major conferences
– Journals are a few years behind, but still can
be useful
• Tech reports from active research groups
– need to know which groups to look up
• Survey / overview papers
– ACM Computing Surveys
– CACM, IEEE Computer, Spectrum
– more technical - IEEE Personal
Communications, …
– newsletters - ACM SIGCOMM, ACM
SIGMOBILE, ...
What’s in a paper [see also how to write papers:
http://www.auburn.edu/~xzq0001/WritePapers-Part 1.ppt]
•
•
•
•
•
•
•
•
•
Abstract
Introduction
Motivation
Problem description
Solution
...
Performance Analysis
Conclusions
Future Work
How to read a paper?
Know why you want to read the paper
• To know what’s going on (e.g., scanning
proceedings)
– title, authors, abstract
• Papers in your broad research area
– introduction, motivation, solution description,
summary, conclusions
– sometimes reading more details useful, but
not always
• Papers you may want to improve on
– read entire paper carefully
What to note?
• Authors and research group
– Need to know where to look for a paper on
particular topic
• Theme of the solution
– Should be able to go back to the paper if you
need more info
• Approach to performance evaluation
• Note any shortcomings
So this paper is in print ...
• Be skeptical
• If it sounds too good to be true, it often is
……
More Advice …
Three phases to reading
• Determine if there is anything interesting
at all in the paper.
• Determine which portion of the paper
contains the interesting stuff.
• Should I read the whole paper?
– Read the whole paper if necessary.
Is there anything interesting?
• Ideally, the abstract should tell you this,
but frequently it does not.
• Need to jump about
– Read conclusion
– Read introduction
– Look at the bibliography
– Glance at the TOC (if any)
Which portion contains
interesting stuff?
• A paper typically outlines its organization
at the end of the introduction.
– Use this to determine which portion contains
the exciting stuff.
• Quickly scan the titles of each sections in
the paper.
Read the whole paper
• Read with the following questions in mind
– How can I use this stuff?
– Does this really do what the author claims to
do?
– What if the assumptions and choices that the
author made are discarded (or made invalid)?
– More questions ……
Context and problem statement
• What problems are the author trying to
solve or trying to convince you of?
• Are they important problems?
– Why?
– Why not?
• What is the author’s thesis?
Related work evaluation
• Does the author describe other work in the
field?
• If so, how does this research differ from
the other work?
New idea
• What new idea is the author proposing?
– Architecture
– Algorithm
– Mechanism
– Methodology
– Perspective
• Is the idea useful and practical?
What to evaluate?
• What need to be evaluated to confirm the
worthiness of the new idea?
– Runtime
– Throughput
– Resource utilization
– Model validation
How to evaluate?
• How did the author go about conducting
the evaluation?
– Formalize and prove theorems
– Run simulations
– Artifact design and construction
– Collect traces from existing systems
Was the evaluation correct and
adequate?
• How was the data collection done?
• Do you agree with the analysis of data?
• Do you agree with the conclusions about
the data?
• Do you have any new interpretation of the
data?
• Can you suggest new ways to evaluate
the data?
Assumptions, drawbacks and
extensions
• Can you think of other aspects of the idea
that need to be evaluated?
• Can you think of extensions or
modifications to the idea to improve it?
• How would you evaluate your
improvement?
Assumptions, drawbacks and
extensions
• Can you apply the idea or method of
evaluation to your own project?
• Do the authors make any assumptions
that are not valid or realistic?
• Can you come up with a more general
solution that does not rely on one or more
of the assumptions?
Future work
• Does the author indicate how the work
should be followed up on?
• Does the paper generate new ideas?
• Does the paper implicitly or explicitly
provide a new way of doing other things or
of thinking about problems?
Summary
If you remember nothing else:
• Where to search papers
• Scan papers
• Read papers with questions in mind
• Download the slides from
http://www.auburn.edu/~xzq0001/How to Read Papers - Qin.ppt
http://www.auburn.edu/~xzq0001
Questions