So* You Want to Produce a Literature Review?

So, You Want to Play the
Publish or Perish Game…
What I have Learned
from Software Engineering
Daniel Amyot
[email protected]
E-Business Seminars, February 2, 2017
Why Publish?
• Expected from my program…
• Expected by my supervisor… got no choice!
• I just want to graduate!
So, You Want to Play the Publish or Perish Game…
p.2
Why Publish, Really?
• Results that remain hidden have now value
• You build on the work of others, and others will
build on your work
– Community, continuity
• To get feedback on your work
– Good progress indicator, “baby steps”
– Ph.D. Symposium: excellent for thesis proposal!
• To get some external validation
– Very good to have in a thesis… but careful with different
departmental cultures!
• Fame and glory
– Get noticed! Reputation helps with career and grants
So, You Want to Play the Publish or Perish Game…
p.3
But What Do I Know About
Publishing Papers?
• Still alive…
• As a student: two theses (uOttawa!)
• (Co-)author of several papers
– 8 book chapters, 36 journals, 125
conferences/workshops, 3 patents
• Supervisor: 2 Post-Doc, 8 Ph.D., 32 M.Sc. completed
• (Co-)editor:
– 10 conference proceedings (IEEE CS, Springer, IOS
Press)
– 5 international standards
• And still learning!
So, You Want to Play the Publish or Perish Game…
p.4
First:
Know The Game
So, You Want to Play the Publish or Perish Game…
p.5
Who Are the Actors Involved?
• Authors (students, researchers, professors, …)
• Reviewers
• For conferences:
– Program Committee/Board
– Program Chair(s)
– Steering Committee
• For journals:
– Editorial Board
– Editors and Associate Editors
• Typesetters
• Publishers
• Buyers (e.g., libraries)
• Readers
• …
So, You Want to Play the Publish or Perish Game…
p.6
What Is The Publication Process?
• Who does what when?
• Many types
– Open, single-blind, or double-blind
– Single round, multiple rounds (with rebuttal, or new
versions)
– Abstracts only, or full papers
– Abstract first: to gauge interest, to start the
assignment process to reviewers, or to make a first
cut at the selection
• Very dependent on the conference/journal
• Get to know it for your target venue before submitting!
So, You Want to Play the Publish or Perish Game…
p.7
Sample Journal
Process (SCIRP)
http://www.scirp.org/about
Us/ForAuthors.aspx
Another Journal
Process
(Elsevier)
https://www.elsevier.com
/reviewers/what-is-peerreview
So, You Want to Play the Publish or Perish Game…
p.9
Publication Types and Formats:
All Have Their Role to Play
•
•
•
•
•
•
•
•
•
•
•
•
Book review
Poster
Abstract
Short/position/vision paper
Doctoral symposium paper
Workshop paper
Conference paper
Journal paper
Book chapter
Invited paper
Thesis
Book…
• How much visibility and
impact do you want?
• How much time do you have?
• What is the quality of your
results/validation?
• How much space do you need?
• How long can you wait?
• What quality of feedback do
you want?
• What tolerance to risk and
rejection do you have?
• How much interaction do you
expect?
So, You Want to Play the Publish or Perish Game…
p.10
There Are Also Non-Peer-Reviewed
Options
• Newsletters, Magazines, Bulletins
• Blogs
• Professional social media
– Academia.edu
– LinkedIn.com
– Slideshare.com
– ResearchGate.com
• Indexed self-archives
– https://arxiv.org/
– http://roarmap.eprints.org/
• Web sites: personal or institutional
– http://ruor.uottawa.ca
So, You Want to Play the Publish or Perish Game…
p.11
Do You Actually Have Something
to Publish?
• Borja (2014) suggests having Yes to all these
questions for a good journal publication
–
–
–
–
Have I done something new and interesting?
Is there anything challenging in my work?
Is my work related directly to a current hot topic?
Have I provided solutions to some difficult problems?
• What do you think?
So, You Want to Play the Publish or Perish Game…
p.12
Choose Your Venues Wisely
• Ask around for good conferences and journals
•
•
•
•
– Supervisors can provide guidance, but look around…
Consider the seriousness of
– Sponsoring organizations (ACM, IEEE, …)
– Program committees (who is involved?)
Publisher quality (Elsevier, Springer, …)
Indexes (directly impacts visibility)
– Scopus, Web of Science, Google Scholar, DBLP, …
Impact factor (not just by itself)
So, You Want to Play the Publish or Perish Game…
p.13
Be Careful!
• Do not target bad conferences/journals for the
sake of increasing your number of publications
– Hurtful to your reputation
– Feedback will not be useful anyway
– Papers are meant to be read, not be write-only!
• Do not get distracted by unfocused emails/spam
advertising bad conferences and journals
So, You Want to Play the Publish or Perish Game…
p.14
How Not To Do It
So, You Want to Play the Publish or Perish Game…
p.15
The Positive and the Negative
Approaches to Venue Selection
• Positive
–
–
–
–
–
Look at evidence or rankings
SCImago Journal & Country Rank
CORE rankings (Australia)
Publish or Perish
Google Scholar Metrics
• Negative
–
–
–
–
Predatory open access publishing (Wikipedia)
All Fake Journals (blog)
Fake Research Journal Publishers
No More 'Beall's List‘
So, You Want to Play the Publish or Perish Game…
p.16
So, You Want to Play the Publish or Perish Game…
http://www.harzing.com/pop.htm
p.17
Google Scholar Metrics
(uses h5-index)
So, You Want to Play the Publish or Perish Game…
p.18
So, You Want to Play the Publish or Perish Game…
p.19
So, You Want to Play the Publish or Perish Game…
p.20
So, You Want to Play the Publish or Perish Game…
p.21
So, You Want to Play the Publish or Perish Game…
p.22
So, You Want to Play the Publish or Perish Game…
p.23
There are Many Submission
Systems
• Influence the submission and review processes
• You and your co-authors may need to create accounts
• For conferences, many open-source ones are popular
– CyberChair
– EasyChair
– …
• Some will also check the “similarity” of your paper with
others
So, You Want to Play the Publish or Perish Game…
p.24
Conferences: Can You Actually Go?
• You are submitting to a conference? Be there!
– Never submit to a conference where it will be
impossible for you and your co-authors to attend and
present (visa, budget, time…)
– No-show: very bad for your reputation
• Financial Help
–
–
–
–
Your supervisor and department!
FGPS funding (before leaving)
GSAED funding
APTPUO funding
So, You Want to Play the Publish or Perish Game…
p.26
Then:
Play The Game
So, You Want to Play the Publish or Perish Game…
p.27
Learn the Author Guidelines
• Vary considerably from venue to venue
• Most conferences will impose the publisher’s template
• Some journals have templates, others will do the
typesetting themselves
• Expectations might be different between the initial
submission and the final paper
• Instructions about fonts, sizes, figures, references, …
• Anonymity might be required (double-blind reviews)
• Beware of intellectual property issues
• Examples:
– http://www.mcetech.org/
– http://www.springer.com/computer/swe/journal/766
So, You Want to Play the Publish or Perish Game…
p.28
What To Do
• Good writing style
– Get to the point!
• Choose your title wisely
– You want to be found by the reviews of others!
• Context, problem, state of the art, objectives,
methodology, solution, results, validation, impact…
• Abstract: more of an executive summary than a teaser…
• Respect imposed templates/formats, if any
• Have someone review the paper before submitting
– MUST DO if not written in your native language
So, You Want to Play the Publish or Perish Game…
p.29
Example of a Weak Abstract
•
We propose Use Case Maps (UCMs) as a notation for describing
features. UCMs capture functional requirements in terms of causal
scenarios bound to underlying abstract components. This
particular view proved very useful in the description of a wide
range of reactive and telecommunications systems. This paper
presents some of the most interesting constructs and benefits of
the notation in relation to a question on a User Requirements
Notation recently approved by ITU-T Study Group 10, which will
lead to a new Recommendation by 2003. Current research on
UCMs is also discussed.
[Amyot, D.: Use Case Maps as a Feature Description Notation. Language
Constructs for Designing Features. Springer-Verlag. 27-44, 2000]
•
Where are the context, objective, methodology, and results?
So, You Want to Play the Publish or Perish Game…
p.30
Example of a Strong Abstract
•
•
Background: Despite the potential benefits of health information
exchange (HIE) and the two decades of efforts from the Canadian
and the American governments to promote health exchange
projects, failures far outnumber successes. Objective: To better
understand the barriers influencing the adoption and
implementation of inter-organization HIE systems in Canada and
the USA. Method: A systematic literature review was conducted
to examine English-language studies that identified barriers to
HIE in Canada and the USA between 1995 and 2016. Electronic
databases, backward searching and expert consultations were
used. Results: 31 articles have been included. There is a dearth
of publications reported on the HIE barriers in Canada. A total of
33 barriers have been identified. Conclusion: There are
noticeable differences in the barriers reported in these countries.
Privacy concerns and a lack of stakeholder buy-in are recurring
barriers over time in the USA. Low adoption of electronic medical
records is the main barrier in Canada.
[Almoaber, B., Amyot, D.: Barriers to Successful Health Information Exchange
Systems in Canada and the USA - A Systematic Review. Int. J. of Healthcare
Information Systems and Informatics, 12(1), 44-63, 2017]
So, You Want to Play the Publish or Perish Game…
p.31
Common IMRAD format
• Introduction: What did you/others do? Why did you do
it?
• Methods: How did you do it?
• Results: What did you find?
• And
• Discussion: What does it all mean?
• In some domains, journals will actually impose a format
So, You Want to Play the Publish or Perish Game…
p.32
Borja’s Advice (2014)
•
•
•
•
•
•
•
•
•
•
•
Prepare the figures and tables.
Write the Methods.
Write up the Results.
Write the Discussion.
Write a clear Conclusion.
Write a compelling introduction.
Write the Abstract.
Compose a concise and descriptive Title.
Select Keywords for indexing.
Write the Acknowledgements.
Write up the References.
So, You Want to Play the Publish or Perish Game…
p.33
What To Do Too
• References
– Appropriate, consistent style, no missing info
– Reference quality = good indicator of paper quality
– Learn how to use LaTeX or Word, really!
• High-quality images
– Avoid bitmaps, use vectorial formats
• If bitmaps, use a lossless format (PNG, not JPG)
– Do not use Word figures or graphs!
– Copy/paste from Visio, PowerPoint, Excel, …
• But Paste as… Enhanced Metafile
So, You Want to Play the Publish or Perish Game…
p.34
http://www.phdcomics.com/comics.php?f=1821
So, You Want to Play the Publish or Perish Game…
p.35
Common Weak Spots:
Methodology and Validation
• Research Methodology: explicit, with reference(s)
• Validation
– Quantitative and/or qualitative analysis
– Industrial validation (by others) if possible;
demonstrate applicability and pragmatism!
– Be neutral and unbiased. Do not over/under claim!
– Recognize limitations; this is a sign of maturity
• Threats to Validity
– Cover the different categories (internal, external, …)
– Explain how they were mitigated, and highlight the
parts that remain
– Think about them in advance… Be proactive!
So, You Want to Play the Publish or Perish Game…
p.36
Claim Your Contribution!
• The first author matters. The rest, substantially less…
So, You Want to Play the Publish or Perish Game…
p.37
Self Checks (Borja, 2014)
•
Does the paper contain sufficient new material?
•
Is the topic within the scope of the journal?
•
Is it presented concisely and well organized?
•
Are the methods and experiments presented in the way that they
can be replicated again?
•
Are the results presented adequately?
•
Is the discussion relevant, concise and well documented?
•
Are the conclusions supported by the data presented?
•
Is the language acceptable?
•
Are figures and tables adequate and well designed?, are there
information duplicated? Are they too many?
•
Are all references cited in the text included in the references list?
So, You Want to Play the Publish or Perish Game…
p.38
Some Writing Tips
• Andrew Hendry’s, on scientific papers
• Daniel Amyot’s, on theses
So, You Want to Play the Publish or Perish Game…
p.39
If you Use Word…
• Learn how to use paragraph and character styles
properly, and stick to them
– Beware of “intelligent” copy-pasting
• Learn how to use cross references
– Use them for sections, figures, tables, references,
algorithms, etc.
– Define your own styles if needed, with numbered
paragraphs (sequential or hierarchical)
• For long tables, do not split rows across pages, and
repeat the first heading across pages
• Do not trust blindly “suggested corrections”
• Go train yourselves on YouTube!
So, You Want to Play the Publish or Perish Game…
p.40
Plan Some Time
for Integrating
Comments from
Co-Authors
and…
Supervisors!
http://phdcomics.com
/comics.php?f=1531
So, You Want to Play the Publish or Perish Game…
p.41
Address Comments (Journals)
This is important. See these 10 tips!
So, You Want to Play the Publish or Perish Game…
p.42
My Best Advice…
Put Yourself in the Shoes of:
• Your supervisors/advisors
– Who need to ensure minimal quality thresholds
– Please, oh please, make their lives easy!
• The reviewers
– Consider the review criteria, and help them help you!
• The readers!
– In the end, they are the ones to whom you are
addressing your paper.
– Can they actually understand your contributions?
– Similar to the difference between teaching and
learning…
So, You Want to Play the Publish or Perish Game…
p.43
Last Pieces of Advice
• Beware of
– Plagiarism! Use quotes surrounding text that is
replicated in your work (and cite the source
reference properly). Figures that come from
elsewhere must also have their source indicated. You
may actually need to ask permission for using them
– “Incremental” publications
• Do not be discouraged if your paper is rejected
– Journals: seldom works the first time anyway
– Conferences: gamble with reviewers
• But:
– Consider the feedback, integrate, improve, resubmit
– Learn not to repeat these mistakes!
So, You Want to Play the Publish or Perish Game…
p.44
And What If This Was Really
Publish and not Perish?
• Useful, detailed advice at
http://libguides.colorado.edu/publishnotperish
http://www.phdcomics.com/comics/archive.php?comicid=333
So, You Want to Play the Publish or Perish Game…
p.45
Useful References
•
•
•
•
•
Borja, A. (2014) Six things to do before writing your manuscript - Part
1. Elsevier Connect, Career Advice.
Borja, A. (2014) 11 steps to structuring a science paper editors will
take seriously - Part 2. Elsevier Connect, Career Advice.
Elsevier Publishing Tips:
– de Hon, F. (2015) From submission to sharing: the life cycle of an
article. Elsevier Connect.
– Blocken, B. (2017) 10 tips for writing a truly terrible journal
article. Elsevier Connect.
– van Hilten, L. (2015) 3 top tips for responding to reviewer
comments on your manuscript. Elsevier Connect.
Common Errors in Student Research Papers, Experimental Biosciences
Resources, Rice University
Organizing Your Social Sciences Research Paper: Common Grammar
Mistakes, USC Libraries - Research Guides
So, You Want to Play the Publish or Perish Game…
p.46
http://phdcomics.com/