Does Mind Matter?
Reputation for Governance
Rosaria Conte
LABSS/ISTC-CNR
http://labss.istc.cnr,it
Lorentz Center, Leiden
12-16 January 2009
Outline




Aspects and levels of cognition
The role of reputation in governance
Modelling Supraindividual Entity
Reputation (SER)
Conclusions
Lorentz Centre
Aspects and levels of cognition
Mental states

Cognitive systems have mental states




Belief
Goal
Emotion
To act on the world they must accomplish also mental
operations, by manipulating their and others’ mental
states







Memorizing
Learning
Reasoning
Planning
Taking decisions
Imitating
Etc.
Lorentz Centre
Doesn’t mean they are
conscious…
QuickTime™ e un
decompressore TIFF (Non compresso)
sono necessari per visualizzare quest'immagine.
Lorentz Centre
Mental manipulation

Includes



Confrontation: qualitative differences may
be observed among mental states p and q
Reasoning (assert/deny:counterfactual
reasoning)
Nesting: representations of
representations (meta-cognition)
Lorentz Centre
Nesting

Given n types of
mental states, f.i.





B:
• Be = Ego believes that
• Bo = Other believes
that
Beliefs
Goals
Emotions

All sorts of
combinations are
possible, whoever the
holder of the mental
states is:
G
• Ge
• Go

Lorentz Centre
E
• Ee
• Eo
Different from ToM

Meta-cognition is broader than Theory of
Mind,
Not only social beliefs, but also
 Social goals
 Social emotions
 Mental states about own mental states

Lorentz Centre
One level nesting
Mental states belong to the same agent
(Ego)
 To others (Other)

Lorentz Centre
Social mental states
Be
Ge
Ee
Bo
Go
“I believe
you are an
atheist”
“I know
what she
likes…”
Eo
“I
understand
your
feeling”
“I make
“I got her to “I want you
him
want…”
to feel
believe…”
ashamed!”
“I hate your “I fear his
“I like her
creed!”
intention…”
shyness”
Lorentz Centre
Self-representation and
manipulation
Be
Ge
Ee
Be
“I believe I
know…”
“I don’t think I
am in love…”
Ge
“I desperately
want to believe
in God!”
“I don’t know
what I want, but
I do know what I
don’t!”
“I’ll never do it
again. I
promise.”
Ee
“I’m happy to
learn that…”
“I wish I didn’t
want to…”
“I cherish my
longing for
him…”
Lorentz Centre
“I decided I’ll
stop
suffering…”
Multiple levels of nesting


Too many combinations
Interesting with mixed
holders:


QuickTime™ e un
decompressore TIFF (Non compresso)
sono necessari per visualizzare quest'immagine.
You want me to believe
that you are in love with
me, but
I want you not to realize
how I really feel about you
QuickTime™ e un
decompressore TIFF (Non compresso)
sono necessari per visualizzare quest'immagine.
Lorentz Centre
Where to stop ?
QuickTime™ e un
decompressore TIFF (Non compresso)
sono necessari per visualizzare quest'immagine.
Lorentz Centre
Depends on what you are
accounting for…

Essential for
communication
Want
I Want
Want
QuickTime™ e un
decompressore TIFF (Non compresso)
sono necessari per visualizzare quest'immagine.

Suppose we had only
one level nesting: it
would not account for the
difference between


Manipulation
Communication
I Want
Want
QuickTime™ e un
decompressore TIFF (Non compresso)
sono necessari per visualizzare quest'immagine.
Lorentz Centre
Why bother?
QuickTime™ e un
decompressore TIFF (Non compresso)
sono necessari per visualizzare quest'immagine.
MC
Lorentz Centre
Some properties of nesting

Truthvalue is not extensible from one level to
another:



In BeBe,BeBo, BoBe, BoBo
Both may be true or false
But y may have different rutvalues:
• nested belief may be true, but not the metabelief and
• Viceversa

In particular,


Meta-cognition does not inherit the turthvalue of
cognition
No commitment of the meta-belief’s holder on
truthvalue of nested beliefs.
Lorentz Centre
Evaluation

Evaluation (from Conte and Paolucci, 2002) :

a belief about the power of a given entity wrt
someone’s goal(s)
“This tool is good to chop wood”
Entity ei (tool) has the power to achieve agent ai’s (the
speaker’s) goa to have wood chopped.

Truncated evaluation:”This is a good ax”.

Good for what?
• The entity is evaluated against the goal/function it has been
made for,
• which it incorporates
Lorentz Centre
Social evaluation = Image

People are often evaluated against
others’ goals,

which the target of evaluation may adopt:
“Lola is a good company for shopping”

Or not
“Carola is a good partner for gossip: tell her a news,
and soon it will spread throughout the office!”

Image can be truncated:

“Walter is a good chap”
Lorentz Centre
Image’s characters
 Three
characters
 Target
(T)
 Evaluator (E)
 Beneficiary (B)
 Conceptually
distinct,
 Emprically may overllap…
Lorentz Centre
Self-image: E = T
T
=E=B
 need not be
realistic :-)
QuickTime™ e un
decompressore TIFF (Non compresso)
sono necessari per visualizzare quest'immagine.
Lorentz Centre
Who’s B?

B = E:




When T is evaluated against one of T’s goals (T = B),
evaluation is tutorial


T is evaluated against a goal of E’s
evaluation is self-interested (and prudent;
T  E = B: candidates selection for a job
T = B  E: educational evaluation
With no overlapping, evaluation is neutral

T  E  B: standard peer review in science
Lorentz Centre
Reputation:
meta-evaluation


Belief about how T is evaluated by E (often indefinite or
implicit)
4 characters





QuickTime™ e un
decompressore TIFF (Non compresso)
sono necessari per visualizzare quest'immagine.
T
E
B
G (gossiper)
The former 3 may overlap, but not all!



G will always (pretend to) report on others’ evaluation
G does not necessarily share the evaluation
G does not take responsibility over its consequences
Lorentz Centre
Reputation and Image

Complex interplay



G may report on the reputation of x
G may report on his image of x
G may report on both
• These may coincide: “I have the worst possible opinion
about that guy, and I know him to be ill-reputed”
• Or not:
• “I don’t care what people say: I think Diego is a sweetie…”
• “Do you know what they say about Fatima’s last date? I mean
I think he’s great, buti found out he’s known to be a rogue!”
Lorentz Centre
Why bother?

Exchange and cooperation
Partner selection
 Identify and isolate cheaters

Norm-enforcement
 Group formation and maintenance

Lorentz Centre
Image and Reputation:
A Comparison


What are the respective effects?
Hypotheses

In partner selection and norm-enforcement
• Image: insufficient, since social knowledge is acquired only
via direct experience
• But what about image exchange? In NormSim
(Castelfranchi et al., 1998), it was shown to allow normcompliant to compete with cheaters in the same population
• What is the difference between image exchange and
reputation transmission?

In social and cultural evolution: reputation allowed
the enlargement of human settlements (Dunbar,
1998)
Lorentz Centre
The role of reputation in
governance
The system Repage
The main element of the Repage architecture is the
memory, which is composed by

a set of predicates. containing either

a social evaluation, either image,
reputation, shared voice, shared evaluation
or

valued information,

evaluation related from informers, and

outcomes.

contain a tuple of five numbers representing
the evaluation plus a strength value that
indicates the confidence the agent has on this
evaluation.

conceptually organized in a network of
dependencies, specifying which predicates
contribute to the values of others:

each predicate has a set of antecedents
and a set of consequents.

If an antecedent is created, removed, or its
value changes, the predicate is notified,
recalculates its value and notifies the
change to its consequents.
REPAGE runs on a JADE-X platform.
Lorentz Centre
The market scenario
Simulations were run (Paolucci et al., 2007; Quattrociocchi et al., 2008)
with fixed number of sellers and buyers (respectively to 100 and 15).
Goods are represented by a 1-100 valued utility factor (interpreted as
quality, but, at this level of abstraction, could as well represent other
utility factors as quantity, discount, timeliness).
Results were explored with an instrument (Dimensional Fact Modelling) to
extract from simulation findings factors, also in their combinations and
interactions with others, relevant for our reputation theory.
Lorentz Centre
Main findings
QuickTime™ e un
decompressore TIFF (LZW)
sono necessari per visualizzare quest'immagine.
Lorentz Centre
Further directions

Governance
Opinion manipulation and political choice
 Reputation in industrial clusters
Beyond
the
 Institutional reputation

individual
Lorentz Centre
Monitoring institutions:
The case of last Italian election
(from Quattrociocchi et al., in preparation)
QuickTime™ e un
decompressore TIFF (LZW)
sono necessari per visualizzare quest'immagine.
Lorentz Centre
Media informational cheating…
QuickTime™ e un
decompressore TIFF (LZW)
sono necessari per visualizzare quest'immagine.
Lorentz Centre
Social perception
QuickTime™ e un
decompressore TIFF (LZW)
sono necessari per visualizzare quest'immagine.
Lorentz Centre
What has happened?


Quality collapses: people buy (vote for) security
champions, which they don’t need (i.e., lemmons)
At what price? Probably giving up




Welfare state, especially R&D investment
Conflict-of-interest legislation
Pursuit of bribery.
Hence



high information cheating through the media
feeds gossip
In turn reinforcing information cheating
Lorentz Centre
A puzzle?

Social perception
follows the trend
of lies:
People buy
lemmons,
assuming that it is
gold….

???

QuickTime™ e un
decompressore TIFF (LZW)
sono necessari per visualizzare quest'immagine.
Lorentz Centre
Questions for policy makers

How reduce media
informational cheating on social
perception?

How reduce the effects of
media on people’s private
communication?
Lorentz Centre
Question for the social scientist

When and why informational
cheating produces a self-fulling
prophecy?
Lorentz Centre
(e-)Governance



The role of reputation networks in industrial
clusters (see SOCRATE,
http://socrate.istc.cnr.it/socrate )
Reputation technology on the Internet (see
eREP,
http://megatron.iiia.csic.es/eRep/?q=tracker )
Evaluation by results: reputation of institutions
and public administration
Lorentz Centre
Reputation on the Internet:
main results
(From eREP first deliverable, at
http://megatron.iiia.csic.es/eRep/?q=node/37 )

Image and reputation imply different degree of




Commitment
Repsonsibility on consequences of communication
Hence, conditions of communication may force one or the other with different
outputs
When

no reputation report is allowed, and one or more of the following conditions holds
• Feedback is target accessible
• Choice of recipinet is unfeasible
• Feedback is not anonymous


Underprovision and overrating are likely (courtesy equilibrium)
In the opposite conditions, provision and underrating can be expected (prudence
equilibrium)
Lorentz Centre
Modelling Supraindividual Entity
Reputation (SER)
Superindividual entity reputation
(SER)

How




model
Implement
Quantify it
Why bother
Lorentz Centre
Modelling SER
(From Conte and Zaccaria, 2009, forthcoming)

Internal (the entity’s)



Global (before its members)
Distributed (of its members)
External (before external agencies)


global (of entity on the market, before users or
clients etc.)
distributed (reputation of members in the external
world, public employees, FIAT workers,
parliamentary, etc.).
Lorentz Centre
Some suggestions

Take into account

difference between I and R: allow for
choosing between
• direct and
• reported on feedback,
• With and without explicit source
Lorentz Centre
Feedback provision: the role of
structural modalities

Activate




Users’ networks for exchanging info about
personal experiences;
Accessible Vs protected feedback ;
Broadcast Vs narrowcast feedback,
With and without choice of recipient,
• Target-accessible only to target
• User-accessibile soltanto a un subset di utenti


Anonymous Vs signed feedback
etc.
Lorentz Centre
Internal and external SER


What do members believe and report on
compared to users?
Multiple external and internal sources:
• from other istitutions, users, pribate or public competitors.
• feedback from social network (accompanying person, family,
etc.; but how aggregate them?).

How do they interact?
Lorentz Centre
Global and distributed SER
Is global SER a sum of members’
reputation?
 How do they interact?
 Members inherit SER, but also
 Affect it
 To what extent?
 Are there critical thresholds?

Lorentz Centre
The circuit of external
reputation





If SE are funded on the grounds of their global
reputation
External global reputation affects external
distributed reputation
External distributed reputation of members
affects external individual reputation
Members manage SER external reputation to
manage their own reputation
SER manages (selects) members to manage
external reputation.
Lorentz Centre
Conclusions




Social evaluation is crucial for
governance
But evaluation and metaevaluation have different impact
They have different advantages
and disadvantages (see
REPAGE results)
Which must be understood :



QuickTime™ e un
decompressore TIFF (Non compresso)
sono necessari per visualizzare quest'immagine.
Conditions favouring either are to
be found out
Impact of either need to be
assessed
But to understand them we need
to model the properties of
complex mental states:
Lorentz Centre