CHI 2008 Proceedings · Works In Progress
April 5-10, 2008 · Florence, Italy
PhotoChat: Communication Support
System based on Sharing Photos and
Notes
Yasuyuki Sumi
Graduate School of Informatics,
Kyoto University
Yoshida-Honmachi, Sakyo-ku,
Kyoto 606-8501, Japan
[email protected]
Jun Ito
Graduate School of Informatics,
Kyoto University
Yoshida-Honmachi, Sakyo-ku,
Kyoto 606-8501, Japan
[email protected]
Toyoaki Nishida
Abstract
This paper proposes PhotoChat, a system that
facilitates communication among users who want to
share experiences by enabling them to share photos
and notes. PhotoChat is designed to be used as a
digital camera and to run on mobile PCs with a camera
module. PhotoChat users can comment on the shared
photos with a pen interface. The data, i.e., photos and
comments, are distributed among PhotoChat users in
real time to enable them to learn others' interests and
to chat easily. In this paper, we show an
implementation of our PhotoChat system and
interaction patterns among PhotoChat users observed
during our experimental evaluations.
Graduate School of Informatics,
Keywords
Kyoto University
Chat on photos, collaborative photo annotation,
communication support system, experience sharing
Yoshida-Honmachi, Sakyo-ku,
Kyoto 606-8501, Japan
[email protected]
ACM Classification Keywords
H5.3. Information interfaces and presentation (e.g.,
HCI): Group and Organization Interfaces
Copyright is held by the author/owner(s).
Introduction
CHI 2008, April 5 – April 10, 2008, Florence, Italy
The recent spread of digital cameras and camera
phones has increased opportunities to record and share
our daily experiences by photos. The huge amount of
ACM 978-1-60558-012-8/08/04.
3237
CHI 2008 Proceedings · Works In Progress
photos that increase daily, however, tend to become
dead storage. The stored photos are not effectively
used to share and create experiential knowledge.
Tools that support ongoing communications in
experience venues, so called experience sharing
communication, are expected. The authors have been
developing a system called PhotoChat to encourage
group members to learn the individual interests of each
other and to facilitate conversations among them by
integrating daily, intuitive methods, i.e., photos and
notes. PhotoChat is software designed to be used on
mobile PCs with camera modules. PhotoChat users can
take photos as with usual digital cameras and write
memos and comment on the photos with a pen
interface. The photos and notes are shared among the
users by a wireless network in real time, and then the
users can discover each other's interests and chat
easily and naturally about the photos.
This paper shows the implementation of PhotoChat and
the interaction patterns among PhotoChat users
observed during deployment at museums, daily
meetings, etc.
April 5-10, 2008 · Florence, Italy
For digital note taking, many works, such as Flatland
[3], have enhanced freehand writing and sketching.
Many works have also supported group collaboration
through viewing and annotating on digital documents
such as papers (e.g., XLibris [4]) and Web documents
(e.g., Anchored conversation [5] and Office Web
discussion [6]). Our aim is to support communications
among users, and the characteristics of our approach
employ photos as a communication ground.
The Lock-on-Chat system [7], which supports
conversations anchored to specific locations of shared
images, is designed to support communication among
the audience at presentation-style conferences. On the
other hand, PhotoChat is designed with the metaphor
of a digital camera; all users can easily take photos to
initiate chats with others. In term of practical
implementation issues, PhotoChat uses ad-hoc wireless
communication without any server to realize robustness
with nomadic usage.
Design and implementation of PhotoChat
input: camera
picture
management
Related works
Our aim is to facilitate instant communications among
people, and the characteristics of our approach employs
photos as a communication ground. The recent
progress of mobile phones enables such communication,
i.e., exchanging daily photos with remote colleagues
and comments on them [1,2]. PhotoChat is designed
as a digital camera augumented with a pen interface
and wireless communication so that its users can easily
take photos and provide notes.
another machine
another machine
another machine
another machine
database
input: tablet
output: display
distribution
control
note
management
figure 1. System configuration of PhotoChat.
3238
CHI 2008 Proceedings · Works In Progress
April 5-10, 2008 · Florence, Italy
machines running PhotoChat by an ad-hoc wireless
network in real time, so that the users can share
photos and chat on them.
Updated:
photo having
unread notes
PhotoChat machines share all photos and notes by
distributing them on wireless peer-to-peer connections
without a data storage server. The system was
implemented with Java and Java Media Framework.
The data stored in the individual machines are photo
data (JPEG), the stroke data of notes, and log data
describing the timing of browsing and writing.
User name & time
photo taken
Color depth
represents
how many
notes written
on photos
Browsing & note mode
Switching
modes
Chosen
photo by
user
Other users
browsing
photo
New:
incoming
photo
Camera mode
figure 2. User interface of PhotoChat system.
System Overview
Figure 1 illustrates the system configuration of
PhotoChat. PhotoChat is software running on mobile
PCs ideally with a camera module and a pen tablet.
Users can take photos as with usual digital cameras
and write and draw on them with the pen interface.
The photos and pen strokes are distributed to other
User Interface
Our user interface has been refined several times based
on experimental deployment. For instance, switching
between “camera” and “browsing & note” modes is
seamlessly done so that the users do not have to
consciously make the switch. As a result, even first
time users can intuitively use it without detailed
instructions, and we observed that even elementary
school children could instantly chat on the photos and
invent their own games with PhotoChat.
As shown in figure 2, most of the PhotoChat display is
laid out for the camera viewfinder and photo
browsing/notes. Thumbnails of all photos taken by
connected users are listed on the left hand of the
display by the time taken. Every thumbnail shows the
user name and the time of the photo. The color depth
behind each thumbnail represents how many notes are
written on the corresponding photo. The thumbnails
also show each user who is browsing (and writing on)
which photos by showing names on the thumbnails so
that the users can easily notice which photo is currently
being discussed.
3239
CHI 2008 Proceedings · Works In Progress
April 5-10, 2008 · Florence, Italy
Hyperlink of Photos
PhotoChat has a function for the hyperlink of photos,
which is a unique ability beyond usual digital cameras
and memo pads. Users can hyperlink between photos
by dragging thumbnails of photos and dropping them
onto another photo or a blank sheet.
Jumping to photo
by clicking
thumbnail
Linking thumbnails by drag and drop
figure 3. Hyperlinking of photos.
Figure 3 is an example of hyperlinking. A PhotoChat
user is summarizing strategies for Japanese chess from
an expert by taking photos and notes and clustering
the strategies with the hyperlink function. Users can
jump to the photo by clicking on the thumbnails. Users
can recursively make hyperlinks that enable
hierarchical and network summarizes of photos that
imply knowledge structures.
Data Sharing based on Ad-hoc Network
In the first protype, we employed a server-client
system in which a server centrally stored all photo and
note data and to which all clients were directly
connected. Preliminary experiments, however,
revealed that the server-client system was not robust
because all the clients depended on the server, so the
server machine had to be kept physically centered
among the clients while moving with PhotoChat users;
that was difficult.
The current implementation employs an ad-hoc
network by wirelessly peer-to-peer connecting among
PhotoChat machines. All PhotoChat machines duplicate
all the photo and note data by conveying as connected
as possible, and then all data diffuse among the
PhotoChat users. More concretely, each PhotoChat
machine regularly broadcasts to find connectable
partners and exchanges the different parts of the data
with them. The ad-hoc network enables us to use
PhotoChat anywhere without servers and wireless
access points.
Interaction pattern among PhotoChat users
We deployed PhotoChat in various opportunities,
including event-type experience sharing such as touring
museums, zoos, participating exhibitions, and
conferences; meeting-type experience sharing such as
daily meetings and lectures; and knowledge sharing in
daily experiences such as sharing tips and know-how of
common facilities in a laboratory. This section
describes interaction patterns among PhotoChat users
observed in the above experiments.
Conversations on Photos
Figure 4 is a typical example of the intended PhotoChat
usage while designing PhotoChat. They show photos
taken by and notes written by several PhotoChat users.
Written notes are translated into English in the Figures.
Users tend to use different pen colors, so we can easily
3240
CHI 2008 Proceedings · Works In Progress
April 5-10, 2008 · Florence, Italy
determine that multiple users are “chatting” about the
photos.
③ This is a “Dashi” wheel
(float for a parade).
① What is this?
② I found it
What’s this?
(><)/
④ Where can
I find it?
⑤ Center of exhibition
area “South Asia”.
An interesting and important characteristic of our
approach is that PhotoChat employs the metaphor of
“graffiti.” Usually, writing graffiti on others' property is
discourteous. PhotoChat, however, encourages such
“rude” behavior so that users have fun with
communication itself.
Ordering Experiential Knowledge
Figure 5 shows an example that summarizes personal
experience by mapping photos based on the museum
floor plan. Generally, notes and scrawls during
experiences tend to be fragmented, and careful note
taking and structuring interrupts concentration on the
experiences. The ordering and mapping of photos on
the PhotoChat system using the hyperlink function are
comparatively simple and easy tasks. Regardless of its
simplicity, hyperlinking is an effective method to
provide semi-structures over the fragments of
knowledge.
figure 4. Question and answer about an exhibit in a museum.
Figure 4 shows a photo where the users are conversing
about an exhibit while in a museum. The photo was
taken and included a note “What is this?” by a user who
liked the decoration of the exhibit. Another user
quickly found the photo on her own PhotoChat and then
the real exhibit. She added a comment: “I found it.
What's this?” on the photo. A researcher in the
museum quickly found the photo and the notes and
responded “This is a Dashi wheel (a float for a
parade).” Recently, many museums are keen to have
dualchannel to facilitate the active participation of
museum visitors and museum facilitators who timely
respond to visitor demands. In such a sense,
PhotoChat is a good tool to mediate between a few
museum facilitators and many visitors.
figure 5. Mapping photos of individual exhibits on floor plan.
Figure 6 shows examples of meeting records. The
authors' group uses PhotoChat at daily meetings and
lectures for recording ideas and facilitating instant
3241
CHI 2008 Proceedings · Works In Progress
April 5-10, 2008 · Florence, Italy
communications among the members. The huge
amount of data collected so far can be roughly
classified into two patterns: cartoon-like illustrations of
meeting situations (figure 6(a)), and screen shots of
presentation slides with written notes (figure 6(b)).
Figure 6(a) captures well the situation and atmosphere
of the meetings, not only recording who says what by
speech balloon. Figure 6(b) is a straightforward way to
record meetings, i.e., taking photos of slides and
scribbling notes on them. This method is conventional
and the most familiar way for members to actively
participate in meetings with paper handouts, even
without PhotoChat. A notable phenomenon when using
PhotoChat is that sharing of roles, e.g., shooting slides,
taking notes about them, and summarizing the photos
with the hyperlink function autonomously emerge
among users.
shared photos with a pen interface. The data, i.e.,
photos and notes, are shared among PhotoChat users
in real time, enabling them to chat on the photos, order
experiential knowledge, and invent new usages by
themselves. Currently, we have started to analyze the
system logs regarding photo-taking, note taking, and
browsing behaviors by users as well as real-world
sensor data such as user location, viewing, and
speaking. We will analyze the relationships between
the log data of PhotoChat usage and real-world sensor
data to deepen our understanding and support of user
interactions.
References
[1] Counts, S. and Fellheimer, E. Supporting social
presence through lightweight photo sharing on and off the
desktop, In Proc. CHI 2004, ACM (2004), 599-606.
[2] Ames, M. and Naaman, M. Why we tag: Motivations
for annotation in mobile and online media, In Proc. CHI
2007, ACM (2007), 971-980.
[3] Mynatt, E.D., Igarashi, T., Edwards, W.K., and
LaMarca, A. Flatland: New dimensions in office whiteboards.
In Proc. CHI'99, ACM (1999), 346-353.
[4] Schilit, B.N., Golovchinsky, G., and Price, M.N.
Beyond paper: supporting active reading with free-form
digital ink annotations. In Proc. CHI'98, ACM (1998), 249256.
(a) Scene illustration with speech balloons.
(b) Ordering presentation slides and notes.
figure 6. Meeting records
Conclusions
This paper presented a system called PhotoChat that
enhances experience sharing communication by sharing
photos and notes. PhotoChat users can write notes on
[5] Churchill, E.F., Trevor, J., Bly, S., Nelson, L., and
Cubranic, D. Anchored conversations: Chatting in the
context of a document. In Proc. CHI 2000, ACM (2000),
454-461.
[6] Brush, A.J.B., Bargeron, D., Grudin, J., and Gupta, A.
Notification for shared annotation of digital documents. In
Proc. CHI 2002, ACM (2002), 89-96.
[7] Nishida, T. and Igarashi, T. Lock-on-Chat: Boosting
anchored conversation and its operation at a technical
conference. In INTERACT 2005, Springer LNCS3585 (2005),
970-973.
3242