Inference in First-Order Logic
Philipp Koehn
16 March 2017
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
A Brief History of Reasoning
450B . C.
322B . C.
1565
1847
1879
1922
1930
1930
1931
1960
1965
Philipp Koehn
Stoics
Aristotle
Cardano
Boole
Frege
Wittgenstein
Gödel
Herbrand
Gödel
Davis/Putnam
Robinson
1
propositional logic, inference (maybe)
“syllogisms” (inference rules), quantifiers
probability theory (propositional logic + uncertainty)
propositional logic (again)
first-order logic
proof by truth tables
∃ complete algorithm for FOL
complete algorithm for FOL (reduce to propositional)
¬∃ complete algorithm for arithmetic
“practical” algorithm for propositional logic
“practical” algorithm for FOL—resolution
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
The Story So Far
2
● Propositional logic
● Subset of prepositional logic: horn clauses
● Inference algorithms
– forward chaining
– backward chaining
– resolution (for full prepositional logic)
● First order logic (FOL)
–
–
–
–
variables
functions
quantifiers
etc.
● Today: inference for first order logic
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Outline
3
● Reducing first-order inference to propositional inference
● Unification
● Generalized Modus Ponens
● Forward and backward chaining
● Logic programming
● Resolution
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
4
reduction to
prepositional inference
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Universal Instantiation
5
● Every instantiation of a universally quantified sentence is entailed by it:
∀v α
S UBST({v/g}, α)
for any variable v and ground term g
● E.g., ∀ x King(x) ∧ Greedy(x) Ô⇒ Evil(x) yields
King(John) ∧ Greedy(John) Ô⇒ Evil(John)
King(Richard) ∧ Greedy(Richard) Ô⇒ Evil(Richard)
King(F ather(John)) ∧ Greedy(F ather(John)) Ô⇒ Evil(F ather(John))
⋮
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Existential Instantiation
6
● For any sentence α, variable v, and constant symbol k
that does not appear elsewhere in the knowledge base:
∃v α
S UBST({v/k}, α)
● E.g., ∃ x Crown(x) ∧ OnHead(x, John) yields
Crown(C1) ∧ OnHead(C1, John)
provided C1 is a new constant symbol, called a Skolem constant
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Instantiation
7
● Universal Instantiation
– can be applied several times to add new sentences
– the new KB is logically equivalent to the old
● Existential Instantiation
– can be applied once to replace the existential sentence
– the new KB is not equivalent to the old
– but is satisfiable iff the old KB was satisfiable
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Reduction to Propositional Inference
8
● Suppose the KB contains just the following:
∀ x King(x) ∧ Greedy(x) Ô⇒ Evil(x)
King(John)
Greedy(John)
Brother(Richard, John)
● Instantiating the universal sentence in all possible ways, we have
King(John) ∧ Greedy(John) Ô⇒ Evil(John)
King(Richard) ∧ Greedy(Richard) Ô⇒ Evil(Richard)
King(John)
Greedy(John)
Brother(Richard, John)
● The new KB is propositionalized: proposition symbols are
King(John), Greedy(John), Evil(John), Brother(Richard, John), etc.
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Reduction to Propositional Inference
9
● Claim: a ground sentence∗ is entailed by new KB iff entailed by original KB
● Claim: every FOL KB can be propositionalized so as to preserve entailment
● Idea: propositionalize KB and query, apply resolution, return result
● Problem: with function symbols, there are infinitely many ground terms,
e.g., F ather(F ather(F ather(John)))
● Theorem: Herbrand (1930). If a sentence α is entailed by an FOL KB,
it is entailed by a finite subset of the propositional KB
● Idea: For n = 0 to ∞ do
create a propositional KB by instantiating with depth-n terms
see if α is entailed by this KB
● Problem: works if α is entailed, loops if α is not entailed
● Theorem: Turing (1936), Church (1936), entailment in FOL is semidecidable
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Practical Problems with Propositionalization 10
● Propositionalization seems to generate lots of irrelevant sentences.
● E.g., from
∀ x King(x) ∧ Greedy(x) Ô⇒ Evil(x)
King(John)
∀ y Greedy(y)
Brother(Richard, John)
it seems obvious that Evil(John), but propositionalization produces lots of facts
such as Greedy(Richard) that are irrelevant
● With p k-ary predicates and n constants, there are p ⋅ nk instantiations
● With function symbols, it gets nuch much worse!
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
11
unification
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Plan
12
● We have the inference rule
– ∀ x King(x) ∧ Greedy(x) Ô⇒ Evil(x)
● We have facts that (partially) match the precondition
– King(John)
– ∀ y Greedy(y)
● We need to match them up with substitutions: θ = {x/John, y/John} works
– unification
– generalized modus ponens
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Unification
13
● U NIFY(α, β) = θ if αθ = βθ
p
Knows(John, x)
Knows(John, x)
Knows(John, x)
Knows(John, x)
Philipp Koehn
q
Knows(John, Jane)
Knows(y, M ary)
Knows(y, M other(y))
Knows(x, M ary)
θ
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Unification
14
● U NIFY(α, β) = θ if αθ = βθ
p
Knows(John, x)
Knows(John, x)
Knows(John, x)
Knows(John, x)
Philipp Koehn
q
Knows(John, Jane)
Knows(y, M ary)
Knows(y, M other(y))
Knows(x, M ary)
θ
{x/Jane}
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Unification
15
● U NIFY(α, β) = θ if αθ = βθ
p
Knows(John, x)
Knows(John, x)
Knows(John, x)
Knows(John, x)
Philipp Koehn
q
Knows(John, Jane)
Knows(y, M ary)
Knows(y, M other(y))
Knows(x, M ary)
θ
{x/Jane}
{x/M ary, y/John}
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Unification
16
● U NIFY(α, β) = θ if αθ = βθ
p
Knows(John, x)
Knows(John, x)
Knows(John, x)
Knows(John, x)
Philipp Koehn
q
Knows(John, Jane)
Knows(y, M ary)
Knows(y, M other(y))
Knows(x, M ary)
θ
{x/Jane}
{x/M ary, y/John}
{y/John, x/M other(John)}
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Unification
17
● U NIFY(α, β) = θ if αθ = βθ
p
Knows(John, x)
Knows(John, x)
Knows(John, x)
Knows(John, x)
q
Knows(John, Jane)
Knows(y, M ary)
Knows(y, M other(y))
Knows(x, M ary)
θ
{x/Jane}
{x/M ary, y/John}
{y/John, x/M other(John)}
f ail
● Standardizing apart eliminates overlap of variables, e.g., Knows(z17, M ary)
Knows(John, x)
Philipp Koehn
Knows(z17, M ary)
{z17/John, x/M ary}
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
18
generalized modus ponens
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Generalized Modus Ponens
19
● Generalized modus ponens used with KB of definite clauses
(exactly one positive literal)
● All variables assumed universally quantified
p1′, p2′, . . . , pn′, (p1 ∧ p2 ∧ . . . ∧ pn ⇒ q)
qθ
where pi′θ = piθ for all i
● Rule:
King(x) ∧ Greedy(x) Ô⇒ Evil(x)
● Precondition of rule:
p1 is King(x)
● Implication:
q is Evil(x)
● Facts:
p1′ is King(John)
● Substitution:
θ is {x/John, y/John}
p2 is Greedy(x)
p2′ is Greedy(y)
⇒ Result of modus ponens: qθ is Evil(John)
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
20
forward chaining
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Example Knowledge
21
● The law says that it is a crime for an American to sell weapons to hostile nations.
The country Nono, an enemy of America, has some missiles, and all of its
missiles were sold to it by Colonel West, who is American.
● Prove that Col. West is a criminal
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Example Knowledge Base
22
● . . . it is a crime for an American to sell weapons to hostile nations:
American(x) ∧ W eapon(y) ∧ Sells(x, y, z) ∧ Hostile(z) Ô⇒ Criminal(x)
● Nono . . . has some missiles, i.e., ∃ x Owns(N ono, x) ∧ M issile(x):
Owns(N ono, M1) and M issile(M1)
● . . . all of its missiles were sold to it by Colonel West
∀ x M issile(x) ∧ Owns(N ono, x) Ô⇒ Sells(W est, x, N ono)
● Missiles are weapons:
M issile(x) ⇒ W eapon(x)
● An enemy of America counts as “hostile”:
Enemy(x, America) Ô⇒ Hostile(x)
● West, who is American . . .
American(W est)
● The country Nono, an enemy of America . . .
Enemy(N ono, America)
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Forward Chaining Proof
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
23
16 March 2017
Forward Chaining Proof
24
(Note: ∀ x M issile(x) ∧ Owns(N ono, x) Ô⇒ Sells(W est, x, N ono))
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Forward Chaining Proof
25
(Note: American(x) ∧ W eapon(y) ∧ Sells(x, y, z) ∧ Hostile(z) Ô⇒ Criminal(x))
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Properties of Forward Chaining
26
● Sound and complete for first-order definite clauses
(proof similar to propositional proof)
● Datalog (1977) = first-order definite clauses + no functions (e.g., crime KB)
Forward chaining terminates for Datalog in poly iterations: at most p ⋅ nk literals
● May not terminate in general if α is not entailed
● This is unavoidable: entailment with definite clauses is semidecidable
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Efficiency of Forward Chaining
27
● Simple observation: no need to match a rule on iteration k
if a premise wasn’t added on iteration k − 1
Ô⇒ match each rule whose premise contains a newly added literal
● Matching itself can be expensive
● Database indexing allows O(1) retrieval of known facts
e.g., query M issile(x) retrieves M issile(M1)
● Matching conjunctive premises against known facts is NP-hard
● Forward chaining is widely used in deductive databases
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Hard Matching Example
28
Diff(wa, nt) ∧ Diff(wa, sa) ∧
Diff(nt, q)Diff(nt, sa) ∧
Diff(q, nsw) ∧ Diff(q, sa) ∧
Diff(nsw, v) ∧ Diff(nsw, sa) ∧
Diff(v, sa) Ô⇒ Colorable()
Diff(Red, Blue) Diff(Red, Green)
Diff(Green, Red) Diff(Green, Blue)
Diff(Blue, Red) Diff(Blue, Green)
● Colorable() is inferred iff the constraint satisfaction problem has a solution
● CSPs include 3SAT as a special case, hence matching is NP-hard
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Forward Chaining Algorithm
29
function FOL-FC-A SK(KB, α) returns a substitution or false
repeat until new is empty
new ← ∅
for each sentence r in KB do
( p 1 ∧ . . . ∧ p n Ô⇒ q) ← S TANDARDIZE -A PART(r)
for each θ such that (p 1 ∧ . . . ∧ p n)θ = (p ′1 ∧ . . . ∧ p ′n)θ
for some p ′1, . . . , p ′n in KB
q ′ ← S UBST(θ, q)
if q ′ is not a renaming of a sentence already in KB or new then do
add q ′ to new
φ ← U NIFY(q ′, α)
if φ is not fail then return φ
add new to KB
return false
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
30
backward chaining
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Backward Chaining
31
● Start with query
● Check if it can be derived by given rules and facts
– apply rules that infer the query
– recurse over pre-conditions
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Backward Chaining Example
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
32
16 March 2017
Backward Chaining Example
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
33
16 March 2017
Backward Chaining Example
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
34
16 March 2017
Backward Chaining Example
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
35
16 March 2017
Backward Chaining Example
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
36
16 March 2017
Backward Chaining Example
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
37
16 March 2017
Backward Chaining Example
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
38
16 March 2017
Properties of Backward Chaining
39
● Depth-first recursive proof search: space is linear in size of proof
● Incomplete due to infinite loops
Ô⇒ fix by checking current goal against every goal on stack
● Inefficient due to repeated subgoals (both success and failure)
Ô⇒ fix using caching of previous results (extra space!)
● Widely used (without improvements!) for logic programming
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Backward Chaining Algorithm
40
function FOL-BC-A SK(KB, goals, θ) returns a set of substitutions
inputs: KB, a knowledge base
goals, a list of conjuncts forming a query (θ already applied)
θ, the current substitution, initially the empty substitution ∅
local variables: answers, a set of substitutions, initially empty
if goals is empty then return {θ}
q ′ ← S UBST(θ, F IRST(goals))
for each sentence r in KB
where S TANDARDIZE -A PART(r) = ( p 1 ∧ . . . ∧ p n ⇒ q)
and θ′ ← U NIFY(q, q ′) succeeds
new goals ← [ p 1, . . . , p n∣R EST(goals)]
answers ← FOL-BC-A SK(KB, new goals, C OMPOSE(θ′, θ)) ∪ answers
return answers
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
41
logic programming
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Logic Programming
42
● Sound bite: computation as inference on logical KBs
1.
2.
3.
4.
5.
6.
7.
Logic programming
Ordinary programming
Identify problem
Assemble information
Tea break
Encode information in KB
Encode problem instance as facts
Ask queries
Find false facts
Identify problem
Assemble information
Figure out solution
Program solution
Encode problem instance as data
Apply program to data
Debug procedural errors
● Should be easier to debug Capital(N ewY ork, U S) than x ∶= x + 2 !
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Prolog
43
● Basis: backward chaining with Horn clauses + bells & whistles
● Widely used in Europe, Japan (basis of 5th Generation project)
● Compilation techniques ⇒ approaching a billion logical inferences per second
● Program = set of clauses = head :- literal1, . . . literaln.
criminal(X) :- american(X), weapon(Y), sells(X,Y,Z), hostile(Z).
missile(M1).
owns(Nono,M1).
sells(West,X,Nono) :- missile(X), owns(Nono,X).
weapon(X) :- missile(X).
hostile(X) :- enemy(X,America).
American(West).
Enemy(Nono,America).
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Prolog Systems
44
● Efficient unification by open coding
● Efficient retrieval of matching clauses by direct linking
● Depth-first, left-to-right backward chaining
● Built-in predicates for arithmetic etc., e.g., X is Y*Z+3
● Closed-world assumption (“negation as failure”)
e.g., given alive(X) :- not dead(X).
alive(joe) succeeds if dead(joe) fails
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
45
resolution
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Resolution: Brief Summary
46
● Full first-order version:
`1 ∨ ⋯ ∨ `k ,
m1 ∨ ⋯ ∨ m n
(`1 ∨ ⋯ ∨ `i−1 ∨ `i+1 ∨ ⋯ ∨ `k ∨ m1 ∨ ⋯ ∨ mj−1 ∨ mj+1 ∨ ⋯ ∨ mn)θ
where U NIFY(`i, ¬mj ) = θ.
● For example,
¬Rich(x) ∨ U nhappy(x)
Rich(Ken)
U nhappy(Ken)
with θ = {x/Ken}
● Apply resolution steps to CN F (KB ∧ ¬α); complete for FOL
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Conversion to CNF
47
Everyone who loves all animals is loved by someone:
∀ x [∀ y Animal(y) Ô⇒ Loves(x, y)] Ô⇒ [∃ y Loves(y, x)]
1. Eliminate biconditionals and implications
∀ x [¬∀ y ¬Animal(y) ∨ Loves(x, y)] ∨ [∃ y Loves(y, x)]
2. Move ¬ inwards: ¬∀ x, p ≡ ∃ x ¬p, ¬∃ x, p ≡ ∀ x ¬p:
∀ x [∃ y ¬(¬Animal(y) ∨ Loves(x, y))] ∨ [∃ y Loves(y, x)]
∀ x [∃ y ¬¬Animal(y) ∧ ¬Loves(x, y)] ∨ [∃ y Loves(y, x)]
∀ x [∃ y Animal(y) ∧ ¬Loves(x, y)] ∨ [∃ y Loves(y, x)]
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Conversion to CNF
48
3. Standardize variables: each quantifier should use a different one
∀ x [∃ y Animal(y) ∧ ¬Loves(x, y)] ∨ [∃ z Loves(z, x)]
4. Skolemize: a more general form of existential instantiation.
Each existential variable is replaced by a Skolem function
of the enclosing universally quantified variables:
∀ x [Animal(F (x)) ∧ ¬Loves(x, F (x))] ∨ Loves(G(x), x)
5. Drop universal quantifiers:
[Animal(F (x)) ∧ ¬Loves(x, F (x))] ∨ Loves(G(x), x)
6. Distribute ∧ over ∨:
[Animal(F (x)) ∨ Loves(G(x), x)] ∧ [¬Loves(x, F (x)) ∨ Loves(G(x), x)]
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Our Previous Example
49
● Rules
– American(x) ∧ W eapon(y) ∧ Sells(x, y, z) ∧ Hostile(z) Ô⇒ Criminal(x)
– M issile(M1) and Owns(N ono, M1)
– ∀ x M issile(x) ∧ Owns(N ono, x) Ô⇒ Sells(W est, x, N ono)
– M issile(x) ⇒ W eapon(x)
– Enemy(x, America) Ô⇒ Hostile(x)
– American(W est)
– Enemy(N ono, America)
● Converted to CNF
– ¬American(x) ∨ ¬W eapon(y) ∨ ¬Sells(x, y, z) ∨ ¬Hostile(z) ∨ Criminal(x)
– M issile(M1) and Owns(N ono, M1)
– ¬M issile(x) ∨ ¬Owns(N ono, x) ∨ Sells(W est, x, N ono)
– ¬M issile(x) ∨ W eapon(x)
– ¬Enemy(x, America) ∨ Hostile(x)
– American(W est)
– Enemy(N ono, America)
● Query: ¬Criminal(W est)
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
16 March 2017
Resolution Proof
Philipp Koehn
Artificial Intelligence: Inference in First-Order Logic
50
16 March 2017