10 (Random) Lessons
on Paper Writing
Rodolfo Pellizzoni
Introduction
• Unfortunately, I only archive the final version of each paper, so I
can not show you a “step-by-step” paper analysis.
• Luckily, I have been writing research papers for 5 years now, and
I had my fair share of ups and downs.
• This is a collection of 10 (random?) lessons that I picked up and
always keep in mind.
• Take them with a grain of salt, I am more interested in starting
some discussion than proving facts.
Lesson 1: Be sure to pick the right
conference / journal
• It is very important to submit to a conference / journal whose
topic and interests closely match the contribution of your paper.
• Even within the same community, different conferences can
have different focus!
– For example, ECRTS is more oriented to theory, RTAS is more oriented to
systems. This also reflects in some way differences between the
community in Europe and the US.
• Whenever possible, try to adjust the focus of the paper based
on the conference.
– Even just a slight reword of abstract and introduction can make a big
difference (see Lesson 3).
• Be careful with special tracks: acceptance ratios are typically not
higher than main track, submit only if the paper is very focused.
Example
• R. Pellizzoni and M. Caccamo: Adaptive Allocation of Software
and Hardware Real-Time Tasks for FPGA-based Embedded
Systems. RTAS 2006.
• Algorithms to allocate relocatable tasks on FPGA, the paper is
actually quite theoretical.
• Initially rejected at RTSS 06 with rather negative scores.
– Major concern for most reviewers: “there is no bus schedulability analysis”.
• Accepted at RTAS with pretty good scores.
– We modify the paper to include more precise implementation details.
– Note we did not address bus schedulability at all (we had no bus
implementation at the time).
Lesson 2: Always check the
program committee
• Reviewers are people. They all have their different opinions
and points of view.
• Whenever you want to submit to a conference, be sure to
check the program committee first and identify possible
reviewers!
• Things you should be careful about:
–
–
–
–
Related work.
Position on controversial issues (see Lesson 6).
What they might be interested in seeing in the paper (see Lesson 8).
Etc.
Lesson 3: Section 1 is
the most important
• The introduction is by far the most important section in the entire
paper, especially for conferences.
• Reviewers are always very busy.
– Grad students do a lot of reviews and the return on investments in
reviewing is poor.
– For conference papers, the author can not answer back so reviewers are
less accountable.
• If a reviewer can reject your paper without reading it all, it saves
time!
• The introduction is the first section they read, so make sure your
paper does not get killed in Section 1.
• 5 years ago I used to write the introduction last. Now it is always
the first section I write.
Example
• Every single paper of mine received comments that could have
been avoided with a better introduction.
• Instead of individual examples, here are some general suggestions:
1. Explain why the problem you solve is relevant and important.
–
By far the most important point. Also the hardest to properly articulate.
Sometimes examples help, sometimes not.
2. Clearly state your contribution and why it is original.
–
No original contribution -> “incremental paper” -> autorejection.
3. Summarize your main assumptions and limitations.
–
People are skeptic that you found the holy grail.
4. Try to add something to encourage the reviewer to read on.
–
This is an art that comes with experience.
Lesson 4: Intuitions are more
important than proofs…
• I consider myself a 50/50 person. That means I do 50% theory and
50% implementation.
• It also means I wrote papers that are full of theorems and lemmas.
• Proofs are a necessary evils.
– We are a community of engineers not mathematicians, so reviewers will
not like a paper just because proofs are elegant.
• Theorem: reviewers do not like to read proofs.
– Following Lesson 3, reviewers are busy. Since furthermore reading proofs
takes a lot of time, the theorem follows.
• Proofs also take a lot of space.
Lesson 4: Intuitions are more
important than proofs…
• Solution: always, always provide a good intuition for your result
before you show the proof.
– If the reviewer is happy with the intuition, your paper will probably be ok
(but see also Lesson 5).
– I always review a submission without reading any proof first. If I paper is
lacking in contribution, there is no further need to read detailed proof.
• You can even move all/some proofs to a technical report if you
do not have the space.
– There is no choice between putting the intuition and putting the detailed
proof in the paper. Intuition always win.
Example
• R. Pellizzoni and M. Caccamo, Impact of Peripheral-Processor
Interference on WCET Analysis of Real-Time Embedded Systems,
Submitted to Transactions on Computers (Major Review).
• Journal version of a previous RTSS conference paper.
• Both papers are full of proofs.
• Conference reviewers complained about the complexity of
Theorem 4, which proves our main algorithm is correct.
– A rather complex proof by induction on a non-intuitive index.
• The real problem is that we did not provide a good enough
intuition for the main algorithm.
• No journal reviewer complained about Theorem 4, even if the
proof is exactly the same.
Lesson 5: … but make sure proofs
are correct
• Intuition is more important, but make sure all your proofs are
formally correct!
• If a conference reviewer finds any error, your paper is on a
fast track to rejection.
• In journal submissions you usually get a chance to correct the
mistake if the result seems reasonable, but do not take that as
an excuse to be careless.
Example
• Again, too many to mention. Reviewers can be picky about
different details, especially regarding notation.
• Some general suggestions:
1. A good notation goes a long way in making a proof readable.
–
–
–
Try to avoid using too many subscripts, superscripts, stars, apexes, etc.
Use notation that is accepted in the community (D is relative deadline,
e/c computation time, p/T period, etc.).
Repeat multiple times what each symbol means.
2. Break long theorems into multiple lemmas.
–
People get discouraged when they see a proof three pages long.
Example
3. The simpler the math, the better it is.
–
–
If you use anything more than standard algebra and calculus, be sure to
provide relevant links. Do not assume that everybody knows X just
because every undergrad must study X in your department.
Some proof schemes tend to annoy people more than others.
Lesson 6: Strong statements
are dangerous…
• Be very careful when you make strong statements about some
research issue: there are people that think otherwise.
• Be especially careful when taking position on some hotly
debated topics in the community, like:
–
–
–
–
–
–
RM vs EDF.
P-fair vs multiprocessor EDF.
Partitioned vs global multiprocessor scheduling.
Hard real-time wireless.
Testing vs static analysis.
Etc. etc. etc.
• Instead of saying “X is black”, say “X is usually black, but in some
cases that are not considered in this paper it is white”.
Example
• R. Pellizzoni and M. Caccamo: M-CASH: A Real-Time Resource
Reclaiming Algorithm for Multiprocessor Platforms. Real-Time
Systems Journal, 2008, Vol. 40(1): 117-147.
• We said that we use multiprocessor EDF because while P-fair
is optimal, it leads to excessive number of preemptions.
• Reviewer commented: “Some of the statements made about
the effects of preemptions are too strong: in fact, there exist
real-time systems that use cached data rather seldomly
(because new data is continually being processed).”
Lesson 7: … but if you are
confident, go for it!
• However, high impact papers are those that successfully
challenge existing preconceptions.
• So do not be shy when you state the main contribution of
your paper!
– If it is somehow controversial, you might have some troubles getting
the paper accepted at first, but it is well worth in term of impact.
– If it is not, you should still stress your contribution so the reviewer gets
more interested in the paper (remember Lesson 3)!
• Just be sure to prove your point well enough; the keyword
here is “successfully challenge”.
Example
• R. Pellizzoni et al.: Coscheduling of CPU and I/O Transactions
in COTS-based Embedded Systems. RTSS 2008.
• The analysis in the paper computes a pattern of arrival times
for cache misses that causes max task WCET increase.
– I was scared because the analysis is built on top of my RTSS 07 paper
and furthermore the obtained WCET algorithm is pretty similar.
• In the introduction, I made the rather strong claim that the
result is very counterintuitive because the worst-case pattern
is the opposite of the classic real-time critical instant (caches
are spread out instead of arriving altogether).
• Two reviewers out of four commented that the analysis is very
interesting and new.
Lesson 8: You can not make
everybody happy
• Different reviewers are looking for different things (remember
Lesson 2).
• You must accept that it is simply impossible to make all
reviewers perfectly happy; as shown in previous lectures, you
are forced to make trade-offs.
– For the same reason, take all reviewers’ comments with a grain of salt.
• The key: two half glasses of water are better than one full and
one empty glass here.
– Just one negative review is enough to kill a conference paper.
– Once your paper is out, if the contribution is solid people will start
citing you anyway.
Example
• Pellizzoni and P. Meredith and M. Caccamo and G. Rosu:
Hardware Runtime Monitoring for Dependable COTS-based
Real-Time Embedded Systems. RTSS 2008.
• Paper is composed by three sections:
– Description of the main idea and monitoring theory (main part).
– Test case (very important to show that this new idea works).
– FPGA implementation (short because it is not the core of the paper,
but required to show that we implemented it for real).
• Submitted to RTSS to a new design & verification track.
• Paper just barely got in.
Example
• Two main issues among reviewers.
• Reviewers 1 & 2 commented that the FPGA implementation
details were boring, and we should have expanded the formal
method part (PTLTL monitor synthesis).
• Reviewer 3 wrote: “… the paper sets out to describe a
hardware solution. Yet there is very little actual hardware
design details.”
Lesson 9: Retry, you’ll
have better luck…
• What you should get out of the previous lessons: you can do a
lot to improve your chances of being accepted, but you also
need some luck.
• If you believe that your contribution is important, you should
not get discourage if your paper is rejected!
– Out of my 12 original papers, 7 got rejected / major reviewed first!
– I managed to publish all 7 of them.
• If you do not believe that your contribution is important, than
you should not have written the paper in the first place!
Lesson 10: … but be careful
where you resubmit!
• Be careful if you submit twice in a row to a conference in the same
area, especially if the community is small like in real-time research.
– Reviewers are people, and people are affected by negative biases.
• If you want to resubmit, be sure to:
– Address all reviewers’ comment to the best of your ability, so the same
reviewer can not reject you twice for the same reason.
– Changing title/abstract/introduction can also help.
• Otherwise, just can go directly for journal.
That’s it people!
… or it should be, but
I have a final Lesson…
Lesson 0
• It all boils down to the following Prime Directive:
Write for the reviewers,
not for yourself!!!