Experimental Linguistics
Anca Dinu
March, 2017
Testing a linguistic hypothesis
• Alternatives:
– By gathering information on it from corpora
(naturally occurring, ‘real’ text or speech) and
than performing qualitative and quantitative
(statistical) analysis;
– By designing a formal experiment with ‘artificial’
data made up by the linguist and performing
qualitative and quantitative (statistical) analysis on
the answers.
Experimental Linguistic
• There are many debates on the distinction between
formal and informal design of experiments. Schütze
and Spouse (2013) list 5 major respects in which
typical informal linguistic judgment gathering tends
to differ from standard practice in psychology:
– relatively few speakers (fewer than ten),
– linguists themselves as the participants,
– relatively impoverished response options (such as just
“acceptable,” “unacceptable,” and perhaps “marginal”),
– relatively few tokens of the structures of interest,
– relatively unsystematic data analysis.
Experimental design example: syntatic
experiments
• In syntactic experiments, the typical minimal data is a
pair of sentences that are compared in order to test
whether some structural difference in otherwise
identical sentences correlates with a difference in
grammaticality / acceptability.
• However, the more interesting setting is to look for
interaction of multiple differences in structure, defined
not by just one factor, but by two or more factors
crossed so that all possible combinations can be tested.
• This type of experimental design is called factorial
design.
Experimental design example: syntatic
experiments
• Multi-factor experiments make it possible to
understand how the factors interact with each other.
• Factorial designs are the best tool an experimenter
has for isolating the factors that could give rise to
relative differences in gramaticallity / acceptability.
Experimental design: prerequisits
• The targeted sentences are mixed with other ‘uninteresting’ sentences (control sentences – used as
baseline), so the subject remains unaware of the
structure investigated and the researcher may have
a reference point for the reliability of the
respondant.
• All sentences are assigned a unique code.
• Each subject sees all sentences exactly one time.
Experimental design: prerequisits
• Each sentence has to be judged one at a time,
without going back or skipping.
• Each questionnaire is printed separately and
presents the sentences in a different random
order.
• Randomization is a standard technique used to
insure that the conditions in one run neither
depend on the conditions of the previous run,
nor predict the conditions in the subsequent
runs.