Evaluating complexity in syntax: a computational
model for a cognitive architecture
Philippe Blache
Laboratoire Parole et Langage
CNRS & Aix-Marseille Université
1 / 46
Situation: Different Kinds of Complexity
I
System vs. Structural Complexity (Dahl, 2004)
System complexity
I
I
I
I
I
I
Structural complexity
I
I
I
I
Number of categories
Depth
Number of rules used to build the
structure
Number of words
Absolute vs. Relative Complexity (Miestamo, 2008)
I
I
I
Number of categories in each domain
Number of features for each categories
Grammar size for each domain
Lexicon size
Average local complexity
Absolute complexity: theory-oriented
Relative complexity: user-dependent (evaluating the difficulties that a human
subject encounters when processing language)
Our perspective : structural relative complexity
Dahl O. (2004) The Growth and Maintenance of Linguistic Complexity, John Benjamins.
Miestamo M. (2008) “Grammatical complexity in a cross-linguistic perspective”, in Language Complexity, John Benjamins.
2 / 46
Difficulty in Psycholinguistics
I
Existing models
I
I
I
I
I
Incomplete Dependency Hypothesis
Dependency Locality Theory
Early Immediate Constituents principle
Activation
However, they fail at:
I
I
Describing language in natural environment
Explaining interaction between sources of information
Gibson E. (1998) “Linguistic complexity: Locality of syntactic dependencies”, Cognition, 68
Gibson, E. (2000) The dependency locality theory: A distance-based theory of linguistic complexity”, in Image, language, brain, MIT Press.
Hawkins J. (2001) ?Why are categories adjacent?, in Journal of Linguistics, 37.
Vasishth S. (2003) ?Quantifying processing difficulty in human sentence parsing : The role of decay, activation, and similarity-based interference?, in Proceedings of The European Cognitive Science Conference 2003
3 / 46
Language processing in the real word
I
New challenges for Linguistics, Psycholinguistics and NLP
I
I
I
Dealing with natural data
Language in its context: spoken language, natural interaction
Issues
I
I
I
Units are not always possible to determine, difficult to categorize (gradience)
Information can be parcimonious
Language processing relies on domain interaction
4 / 46
Language processing in the real word: problems
I
Syntactic information is often partial
I
Inputs can be ungrammatical
5 / 46
Outline
I
A new way of representing syntax
I
Cohesion: a model for syntactic complexity
I
I
How to measure cohesion
Experimenting Cohesion in human language processing: an interplay between
difficulty and facilitation
6 / 46
Part I
Representing syntactic information
7 / 46
Classical Generative Syntax
Language and grammar
I
Derivation
I
Language = set of derived strings
I
Recursively enumerable
Parsing
I
I
I
Finding a derivation
Building a tree
Consequences:
I
I
There exists a complete grammar of the language
The initial system is a complete grammar (acquisition)
8 / 46
Our Proposal: Property Grammars
I
Describing the characteristics of an input (not building a structure)
I
Linguistic statements as constraints
I
Constraints in a computational sense: filtering + instantiating
I
Declarative approach : no specific mechanism but constraint evaluation
I
Basics:
I
Constraints are independent
I
I
I
I
No ranking (contra OT)
Seperate evaluation (contra (GEN)
No hierarchical structure (contra PSG)
Constraints are at the same level (contra DEP)
9 / 46
What kind of syntactic information?
Core syntactic information:
I
Linear precedence
I
Mandatory cooccurrence between two categories
I
Impossible cooccurrence between two categories
I
No repetition of the same category within a construction
I
Dependency between two categories
10 / 46
Linearity
Prec(A, B) : (∀x, y )[(A(x) ∧ B(y ) → y 6≺ x)]
I
Example: nominal construction
Det ≺ Adj Det ≺ N
Adj ≺ N
N ≺ ProR
l
l
the
very
Det ≺ ProR
N ≺ Prep
l
l - &
famous
l -
l
%
reporter
l
+'&
who
the senator atta
11 / 46
Requirement (cooccurrence)
Req(A, B) : (∀x, y )[A(x) → B(y )]
I
I
Example:
V[trans] ⇒ N[obj]
ProR ⇒ N
Det ⇒ N[com]
V[ditrans] ⇒ Prep
Adj ⇒ N
Prep ⇒ N
Relations without government:
(1) a. The most interesting book of the library
b. *A most interesting book of the library
Sup ⇒ Det[def]
r
The
s
r
r
most
r
%
interesting
%%
x
book
r
of the library
12 / 46
Exclusion (cooccurrence restriction)
Excl(A, B) : (∀x)(6 ∃y )[A(x) ∧ B(y )]
I
Examples
I
I
Nominal construction:
Pro ⊗ N N[prop] ⊗ N[com]
N[prop] ⊗ Prep[inf]
Relative construction:
ProR[subj] ⊗ N[subj]
13 / 46
Uniqueness
Uniq(A) : (∀x, y )[A(x) ∧ A(y ) → x ≈ y ]
I
Example (nominal construction):
Uniq = {Det, ProR, Prep[inf], Adv}
u
The
u
book
that
I read
14 / 46
Dependency: the type hierarchy
dep
mod
comp
subj
dep
mod
spec
comp
subj
obj
iobj
xcomp
aux
conj
:
:
:
:
:
:
:
:
:
:
obj
aux
iobj
conj
xcomp
generic dependency relation
modification (typically adjunction)
specification (typically Det-N)
head-complement
subject
direct object
indirect object
other complements (e.g. N − Prep)
auxiliary-verb
conjunction
15 / 46
Dependency
I
Example:
Det ;spec N[com]
Adj ;mod N
ProR ;mod N
spec
mod %
%
interesting
book
mod
The
most
comp
mod
of
"
library
spec
the
I
Syntactic role: N[subj] ;subj V
I
Agreement: Det[agri ] ;spec N[agri ]; Adj[agri ] ;mod N[agri ]
16 / 46
Example: the nominal construction
Det ≺ {Det, Adj, ProR, Prep, N}
N ≺ {Prep, ProR}
Pro ⊗ {Det, Adj, ProR, Prep, N}
N[prop] ⊗ Det
Uniq = {Pro, Det, N, ProR, Prep}
Det ;spec N
Adj ;mod N
ProR ;mod N
Prep ;mod N
Det ⇒ N[com]
{Adj, ProR, Prep} ⇒ N
17 / 46
Syntactic Representation: Constraint Graph
spec
comp
r
l
u
The o
'
/ interesting
7:
mod
r
most
r
l
l
l
mod
(% ! book
8 Z
l
mod
< of Z
l
u
u
the
spec
l
u
&
library
8 E
r
18 / 46
Constraint violation
Contraint graph
l
l
l %*
very
4 old
dc
The
Characterization
+
P =
l , #
book
d 3 8<
d
c
P− =
P+ =
l
Very
l
dc
*
4 old
{
l
the
d
l + book
d 3 @B
c
P− =
{Det ≺ Adj, Det ≺ N, Adv
Adj, Adj
≺
N, Det
N, Adj
;
N, Adv
Adj, Det
⇒
N, Adv
Adj, Adj ⇒ N}
∅
{Det
≺
N, Adv
Adj, Adj
≺ N, Det
N, Adj
;
N, Adv
Adj, Det ⇒ N, Adv
Adj, Adj ⇒ N}
{Det ≺ Adj}
≺
;
;
⇒
≺
;
;
⇒
19 / 46
Part II
Measuring cohesion
20 / 46
Graph-based measures
I
Def. degree = number of edges incident to the vertex
I
Degree of a category in the grammar:
6 Adj
l
l
Det
l
I
'
Prep r
2 ProR g
r
d
r
l
d
r
l
d
l
r
d
Degree of a category in the sentence:
l
l %
l , !
old
book
The
d 27 >
d
c
#*- 12 @ N
deg[gram] (N) = 9
deg[gram] (ProR) = 2
deg[gram] (Adj) = 1
Pro
deg[sent] (N) = 5
deg[sent] (Adj) = 1
deg[sent] (Det) = 0
21 / 46
Category completeness
I
The completeness level of a category depends on the number of relations in
its description
I
This measure also depends on the number of relations for the category in the
grammar
I
Completeness ratio: the higher the number of relations in the grammar is
verified, the higher the completeness value
completeness(cat) =
deg[sent] (cat)
deg[gram] (cat)
22 / 46
Sentence density
I
Measure based on 4 types or properties: uniqueness, requirement,
dependency, linearity
I
Number of possible properties: for each type, the possible properties is the
number of words
I
Density of a construction: the ratio btw the evaluated properties and the
possible properties:
density (sent) =
|properties(sent)|
|words(sent)|
23 / 46
Satisfaction ratio
I
All constraints can be violated
I
A characterization contains both satisfied and violated constraints
I
The “quality” of a construction depends on the ratio satisfied/violated
I
All constraints can be weighted. We note W + (resp. W − the sum of the
weights of the satisfied (resp. violated) constraints :
satisfaction(sent) =
W + −W −
W + +W −
24 / 46
Cohesion function
Given S a sentence, w the set of its words:
cohesion(S) =
|S|
P
completeness(wi ) ∗ density (S) ∗ satisfaction(S)
i=1
25 / 46
Example 1
le c^
oté hystérique un peu de enfin c’est normal tu vois elle souffre et
machin
the hysterical side rather of well it’s normal you see she suffers and that’s it
l
d
d u
y
l
.
le l cot
un peu
enfin
9 < de
= é k d hyst érique
i
r
d
l
c 0V
r
l & l !
est
> j d normal
d
r
l
[tu vois]
elleY
d '
souffre
:
l
d
r
; et d
machin
d
26 / 46
Example 2
the dog apparently knew perfectly –
– the area
and hmm when we left
the dog decided to follow us
le chien apparemment connaissait parfaitement –
– le coin
et euh quand on est partis
le chien a décidé de nous suivre
le
r
l % chien
< `
d
l
d
'
apparemment
d
*
l
connaissait
57
fe
parfaitement
d
l
r
euh
quand
Z
onY
d
&
l - coin
=
d
d
r
l
et
le
est
? b
l %
partis
D
r
d
r
le
r
l $ l
chien
= Y
> aW
d
d
d
l %
l &
d écid
< X é
; de a
r
d
r
d
l
l &!
nous p r suivre
;
d
d
r
27 / 46
Evaluation
Cat
Det
N
Adj
Adv
Prep
Pro
Conj
Aux
V
Conj
Degree-gram
0
34
11
17
31
4
0
8
7
21
Word
le
côté
hystérique
un
peu
de
enfin
c’
est
normal
tu vois
elle
souffre
et
machin
Deg-sent
0
5
3
0
0
2
0
1
3
2
0
1
2
2
0
Deg-gram
0
34
11
0
17
31
17
4
7
11
0
4
7
0
17
Completeness
0
0.15
0.27
0
0
0.06
0
0.25
0.43
0.18
0
0.25
0.29
0
0
28 / 46
Evaluation
d
d
le
l
r
|
-1 cot
é
%
l
i
d
v
l
hyst érique
d
Words
15
21
Sent. 1
Sent. 2
Constraints
19
38
6 de
un peu
enfin
ld
l
Completeness
0.13
0.17
Density
1.18
1.80
Cohesion
0.15
0.32
l
r
l
le
d
@ [
chien
#
apparemment
d
l
9; ab
connaissait
d
&
parfaitement
le
l
+ " coin
BA
d
d
l
d
r
r
29 / 46
Part III
Brain activity in syntactic processing: evoked
potentials
30 / 46
Event-Related Potentials
Study in Event-Related Potential (ERP)
I
ERP: brain response (electrophysiological) to a stimulus
I
ERP Components: series of positive and negative voltage deflections
31 / 46
Event-Related Potentials: N400
N400 amplitude reflects the effort of integrating a word into the current context
They wanted to make the hotel look more like a tropical resort. So along the driveway
they planted rows of .......
32 / 46
ERP and grammaticality
I
ERP enhanced for ungrammatical sequences:
I
I
I
Early left-anterior negativity (ELAN)
Late posterior positivity (P600)
Syntactic Mismatch Negativity (sMMN)
33 / 46
Mismatch Negativity
I
Early attention-independent brain response
I
MMN: brain reflex of acoustic change detection
I
Reflect the existence of cortical memory networks
34 / 46
Syntactic MMN
Pulvermüller, F. & Assadollahi, R. (2007) Grammar or serial order?: Discrete combinatorial brain mechanisms reflected by the syntactic Mismatch
Negativity, Journal of Cognitive Neuroscience, 19 (6)
I
Larger if the word is presented in an ungrammatical context
35 / 46
An experimental approach
I
Experiment: difference in brain responses between frequent and unfrequent,
grammatical and ungrammatical sequences sequences
I
Predictions:
I
Probability approach: a rare grammatical string elicits a brain response
comparable to a rare ungrammatical string
I
Combinatorial approach: same degree for rare and common grammatical
strings with the same syntactic structure
36 / 46
An experimental approach
Material:
Rare
Frequent
Gram.
der Wut
die Wut /
der Mut
der Mut (the courage): singular, masculine, nominative
die Wut (the rage): singular, feminine, nominative and accusative
Ungram.
*die Mut
-
I The MMN occurred to rare ungrammatical strings, but not to rare or frequent grammatical
strings
I The syntactic MMN therefore reflected grammaticality, but not string probability
37 / 46
MMN: conclusions
I
There is a rule-based syntactic processing
I
It is an early brain response
I
It is automatic (attention-indepedent)
I
It does not depend from other linguistic domains (esp. no semantic)
38 / 46
Part IV
Difficulty and Compensation: the Case of
Idiom Processing
39 / 46
The situation
Idiom: multiword expression with a figurative meaning separate from the literal
meaning
Examples
I
Decomposable idioms (variables)
‘‘let the cat out of the bag’’
I
Non-decomposable idioms (opaque semantics, no variability)
‘‘spill the beans’’, ‘‘kick the bucket’’
Experimental perspective
I
Idioms are read faster
I
Idioms are related with specific brain activities
I
Two different models according to the way they are processed
40 / 46
Non-compositional models (lexical look-up models)
Main ideas
I
Idioms are stored like long words in memory
I
Meaning: direct memory retrieval
I
Meaning retrieval and linguistic processing are to some extent independent
41 / 46
Experimental design
Material
I
Idiom (IDNV)
Paul a une idée derrière la tête depuis ce matin
I
Idiome with violation (IDV)
Paul a une idée derrière le tête depuis ce matin
I
Control (CTRNV)
Paul a une douleur derrière la nuque depuis ce matin
I
Control with violation (CTRV)
Paul a une douleur derrière le nuque depuis ce matin
Specific position to study
I
Recognition point (RP)
I
Modified word, where the violation is introduced (MM)
I
Detection word, where the violation is detected (MD)
42 / 46
Results: recognition point
I
I
Different processing idiom vs. control
More positive amplitude in the P300 and N400 windows for idioms:
facilitation
43 / 46
Results: modified word
I
I
More negative N400 for violated idiom (IDV) than non violated (IDNV):
surprisal et the unexpected (modified) word for idioms
No significative P600 : no repair
44 / 46
Results: detection word
I
I
Small N400, small P600 for the violated control
Positive deflection for IDV (wrt IDNV) at N400+P600 : repair
45 / 46
Conclusion
What can be done with constraints
I
Describing whatever the input (including ill-formed)
I
Measuring structural complexity
Complexity Model : a Cognitive Perspective
I
An interplay between difficulty and facilitation
I
An interaction between different sources of information
I
Complexity depends on the quantity of information to reduce the search space
46 / 46