September 26 2016
LECTURE 6,
INTERACTION STYLES
Human Computer Interaction 2016, fjv
1
Recapitulation Lecture #5
• Computer elements
– Interaction devices
– Focus on input
• Paradigms , Shifts
– From Batch
– To Ubiquitous
• Interaction models
– Norman
– Abowd & Beale
Human Computer Interaction 2016, fjv
2
Abowd & Beale vs. Norman
• Norman 7 stages:
– Helps understanding interaction
• Abowd & Beale:
– Translation between languages
3
– (user-input-core-output)
S
core
– 4 translations
2
• Articulation (1), Performance (2)
Presentation (3), Observation (4)
O
output
4
U
task
I
1
input
Simple: Virtual Reality, articulaton is direct
Complex: Control panels, e.g. Lecture Room lighting
Human Computer Interaction 2016, fjv
3
Example Haptic IO
Human Computer Interaction
4
2016, fjv
Example Haptic IO
Human Computer Interaction 2016, fjv
5
Focus on Haptic Feedback
• Epidural Simulator from Yantric
• Haptic IO-device mounted in mannequin
• Interfaced with 3D models
– Feedback
– Visibility
– Mental model (building of)
• Mental model is built from the combination of
haptic force feedback and 3D visualization
• Metaphore is very direct, i.e. a patient
Human Computer Interaction 2016, fjv
6
Metaphors
• The book metaphor
– Strength and weakness
• The desktop metaphor
– History and examples
• The internet metaphor
– Strength and weakness
• The spatial metaphor
– In development, examples
• The social metaphor
– In development, example
Human Computer Interaction 2016, fjv
7
Internet Metaphor
• Creating metaphor has come “after the fact”
–
–
–
–
Internet is a phone
Internet is a highway
Internet is surfing
Internet is a meadow - browsing
• Good interface Metaphors can not be applied to preexisting technologies
• So, Start with a good Metaphor from the beginning
• Email:
– Transition of “mail”
– Has an address
– Send, Received, Forwarded, Redirected
Human Computer Interaction 2016, fjv
8
Spatial Metaphor
• Humans are adapted to 3D space
– Spatial metaphor results in natural interaction
• Access functionality by acting in realistic space
– The normal desktop (2.5D)
– Gaming (3D virtual world: e.g. Quake, 2nd life)
• Issue in information organization
– Virtual space same problems a Real Space
• Use related problems in improvement of
technology
Human Computer Interaction 2016, fjv
9
Beyond the desktop …
Robertson, G. et al.
"The Task Gallery: A 3D Window Manager."
In Proceedings of CHI 2000
Human Computer Interaction 2016, fjv
10
3D interactions …
Introduction of new metaphor's
gallery, rooms, palette (maze)
Human Computer Interaction 2016, fjv
11
3D Navigation
How well is the mapping of the keyboard realized?
A modern approach to space navigation
Human Computer Interaction 2016, fjv
12
Spatial Metaphors …
• 3D interaction in the (G)UI is often thought about
and implementations are now finding their way to
applications and users.
• Jared White: ” A real 3D interface may be many years
from fruition, but use of computers will be
unnecessarily restricted in both cognitive and
functional sense, and also in the social sense as long
as we are living in a flat, two-dimensional virtual
world.” ,
August 2004, adapted by fjv.
Human Computer Interaction 2016, fjv
13
3D Interaction & Spatial Metaphor
• Win3D, ClockWise
– Microsoft Windows desktop
– 1999 – 2003. See examples
• Looking Glass Project, SUN Microsystems
– Linux and Solaris desktop
– 2004 …, See examples
• Croquet Project
– What if .., desktop independent Open Source
– 2004 …, Cf. groupware
Human Computer Interaction 2016, fjv
14
Modernized 3D desktop Metaphor
• Simplex
–
–
–
–
–
Continuously updated
Office zone
Multi-media zone
Games zone
• Kids World
– Play
– No learning curve
– Fun
• Not immersive
Human Computer Interaction 2016, fjv
15
Human Computer Interaction 2016, fjv
16
Human Computer Interaction 2016, fjv
17
Human Computer Interaction 2016, fjv
18
Human Computer Interaction 2016, fjv
19
3D DeskTop Project
• Looking Glass
• Translucent windows!
– See what you are doing where
– Building-Rooms paradigm
• Interact with 3D objects
– Multi media driven
– Object~Action mental model
– Modern approach to objects …
• See also MacOS-X interface
Human Computer Interaction 2016, fjv
20
Human Computer Interaction 2016, fjv
21
Human Computer Interaction 2016, fjv
22
Human Computer Interaction 2016, fjv
23
Human Computer Interaction 2016, fjv
24
3D Interfaces …
Human Computer Interaction 2016, fjv
25
3D Interfaces …
Human Computer Interaction 2016, fjv
26
Other Spatial interactions ...
Make interactive objects physical ...
Human Computer Interaction 2016, fjv
27
Spatial (3D) Interaction
• Increasingly present in a lot of interfaces
– iTunes
– iPod
– 3D and pseudo 3D interaction
• Interaction with objects.
–
–
–
–
Wii
Leap Motion
Kinect
Myo
• Hypertext suggest spatial relations
• Relates to basic HCI metaphores
Human Computer Interaction 2016, fjv
28
Social Metaphor
• Modeling interface to rules and forms of social
behavior
– Related to spatial metaphor (environment)
• No concrete object/concept
– Not functionality but Style/Form
– Social agents
• Social Interaction is often collaborative
– Human
– Human
Human
Artificial agents
• Public virtual spaces, or Digital Hangout
• Relates to basic HCI metaphores
Human Computer Interaction 2016, fjv
29
Social Metaphor
2nd life, a gathering of avatars – a gathering of identities
Human Computer Interaction 2016, fjv
30
Mismatched Metaphors (1)
From the Hall of Shame
• The use of the trashcan to eject the floppy disk…?
• The trashcan on the desktop … ?
Human Computer Interaction 2016, fjv
31
Mismatched Metaphors (2)
From the Hall of Shame
• VCR buttons to control a printer …?
Human Computer Interaction 2016, fjv
32
Basic Level HCI Metaphores
• Direct Manipulation Metaphore:
– DATA are Physical OBJECTS
• Navigation Metaphore
– DATA is in SPACE
– Distances have a meaning.
• Human Interaction Metaphore:
– COMPUTERS are PEOPLE
– Connect with People through Computer networks
Human Computer Interaction 2016, fjv
33
The Metaphor for the iPad ?
• What is your metaphor for your iPad?
– My metaphor is brain (external)
– My tablet metaphor would be like Jamie's Torch.
– Left it in office once, I was bereft,
daughter said its like I'd left her there!
– Mobile shopper ;-)
– Definitely "life partner" ;-)
– A scrapbook
– It’s my “daily computer.” Once or twice a week I’ll
unsheath the MacBook Air for something specific
– Mine would be a window into the world or a bridge to
communicate with others via social networks
Human Computer Interaction 2016, fjv
34
Users & Designers
Metaphors
Graphics
Icons
Language
Documentation
Tutorials
Human Computer Interaction 2016, fjv
35
Features of Systems Image
• Familiar to users (see lecture 4)
– Matches way they think about domain
– Preferably based on a concrete metaphor
– Coherent
•
•
•
•
•
Supports learning by exploration
Hides system model (transparency)
Reflects current status - changes are notified
Supports retention (memorability)
Reduces need for training
Human Computer Interaction 2016, fjv
36
Therefore …
•
Sloan’s Law:
The changes in new designs have to be
dramatically attractive so that unease
about the former design will develop …
•
This law is about a developing a satisfying
Systems Image!
Human Computer Interaction 2016, fjv
37
Paradigm Shifts …
Human Computer Interaction 2016, fjv
38
Yet another Paradigm ...
The Internet of Things (IoT)
Human Computer Interaction 2016, fjv
39
Internet of Things
• “A Thing” connects to Internet
• “A Thing” participates with tasks
• Intelligent environment.
Consequence of Ubiqutous / Cloud Computing
• Calls for new way of design
– Service Design
– What are the HCI aspects to address – Debate.
Human Computer Interaction 2016, fjv
40
Design for IoT
• Key Characteristics (..., to develop an idea)
– Intelligence, Hardware/Software
– Connectivity, Wifi, Accessibility, Compatibility
– Sensing, Experience awareness
– Expressing, Interaction of Thing with Human
– Energy, Power efficiency and sustainability
– Safety, Person, personal data, physical well-being
• Sevice Design is a manner to solve complexity
of connecting with multiple “things” all in a
different way
Human Computer Interaction 2016, fjv
41
The IoT in the living room ...
Human Computer Interaction 2016, fjv
42
IoT, HCI design considerations
•
•
•
•
•
•
•
Rule #1 – Notify Users of Significant Changes
Rule #2 – Be Thorough with Vulnerability Reports
Rule #3 – Give Humans Veto Power
Rule #4 – Provide Method for Prompt and Easy Updates
Rule #5 – Provide a Method for Audit Logging
Rule #6 – Authenticate Input
Rule #7 – Have an Exit Plan
These issues are (2016) being considered
important to arrange in order to have a
successfull, safe and stable IoT.
Most of them are user related.
Human Computer Interaction 2016, fjv
43
INTERACTION STYLES
Human Computer Interaction 2016, fjv
44
Interaction Techniques
• An interaction technique is a way of using a
physical input device to perform a generic task
in a human-computer dialogue. Foley et al. [1990]
technical definition :
• An interaction technique is a way to carry out
an interactive task. It is defined in the binding,
sequencing, and functional levels, and it is
based on using a set of input and output
devices or technologies.
Human Computer Interaction 2016, fjv
45
Five Primary Interaction Styles
Ben Shneiderman 1998
• Direct manipulation
• Menu selection
• Form filling
• Command language
• Natural language
– These are all in response to the baseline of command
languages
– All types not appropriate for all tasks
– Often useful to blend interactions styles in one UI
• We restrict to standard interaction (on a desktop)
Human Computer Interaction 2016, fjv
46
Interaction Styles & Categories
• Describe form of communication
between user and computer
• Linguistic styles
– Command line
– (natural) Speech
• Key-Modal styles
–
–
–
–
Function keys
Q & A interaction
Menu driven
Form Fill-in
• Direct manipulation
– GUI
– Forms (Form Fill-in)
Human Computer Interaction 2016, fjv
47
Command Language
• Oldest style of interaction
• Instructions are expressed to computer directly
• Parsing or Direct Execution
• Communication purely textual
• Options:
•
•
•
•
•
Single characters
Whole words
Abbreviations (e.g. cd, ftp)
Function keys
Combinations of the above
Human Computer Interaction 2016, fjv
48
Example Command Language
CLI: command line interface
Human Computer Interaction 2016, fjv
49
Example Command Line Interface (CLI)
cat, ls, clear, ctrl-d, > ; there is lot to know to make this work ...
Human Computer Interaction 2016, fjv
50
Example Command Line Interface
Human Computer Interaction 2016, fjv
51
Issues in Command Language
• Issues:
– # of keystrokes required to complete command
– burden on (operator) memory
• Main characteristics:
– grammar required
• limited set of commands
• subset of natural language
• syntax often of form “action - object”
Human Computer Interaction 2016, fjv
52
Guidelines Command Language
• Meaningful commands & command names
– based on task analysis
•
•
•
•
Meaningful command use consistent abbreviations
Use consistent CL format (syntax)
Follow ‘de facto’ standards
Provide way to execute sets of commands
– batch processes, Pipelines
• Provide online help, feedback, undo, replay, etc
• Redundancy is okay (provides customizability)
• Use extra options to alter behavior slightly
– reduce memory load
– e.g unix -r option for 10 commands (11 instead of 20)
•
Human Computer Interaction 2016, fjv
53
Command Language
From literature:
Advantages
•
•
•
•
Disadvantages
flexible
appeals to “power” users
allows user initiative
allows convenient creation
of macros
• cheap
• low visibility
• requires substantial training
and memorization
• not useful for “novice”
users
• poor error handling (relative)
Human Computer Interaction 2016, fjv
54
Text-based Natural Language
• Alan Turing: Turing Test
– No difference experienced human/computer
• HEX: Chatterbot
Relevant characteristics
– Convenience (familiar words)
– Expressive power (combinations)
– Ambiguity
• “The house was built by the river by the local council”
– Context
• Cleaning lady earns € 250,
Managing director € 250(.000,-)
Human Computer Interaction 2016, fjv
55
Human Computer 56
Interaction 2016, fjv
Natural Language
LUIs
• Continuous speech recognition (CSR)
• Natural language understanding
– common already
– often narrow in focus (specialized)
• Query languages almost natural language
– grammar limited
Human Computer Interaction 2016, fjv
57
Voice Output
• Has to do with Computer-Human Interaction
Synthesis of voice
• BT
– VODIS: Voice Operated Database Inquiry System
• KPN Telecom
– DIAL 118 phone number request
– Computer operated answer
• Sound-Devices/Cards have Speech Synthesis
– Time indication system
– Read aloud
• New interfaces use sound/voice for instruction
Human Computer Interaction 2016, fjv
58
Example Voice Synthesis (in)
Human Computer Interaction 2016, fjv
59
Example Voice Synthesis (in)
Human Computer Interaction 2016, fjv
60
Example Voice Synthesis (out)
Human Computer Interaction 2016, fjv
61
Example Voice Synthesis (out)
Human Computer Interaction 2016, fjv
62
Example Voice Synthesis (out)
Human Computer Interaction 2016, fjv
63
Natural Language
From literature:
Advantages
Disadvantages
• natural to specify
(no need to learn syntax)
• less intimidating for novices
• can not be applied generally
yet
• requires clarification dialog
• may require more keystrokes
(unless spoken)
• unpredictable results
• Recently, progress !
• Fits mobile devices
Human Computer Interaction 2016, fjv
64
Key-Modal Styles
• Require Input by user/operator
• Question and Answer
• Function key interaction
• Menus
Human Computer Interaction 2016, fjv
65
Function-Key Interaction
Function keys:
• Term covers use of number of special input devices
used for interaction with a machine e.g.
–
–
–
–
numeric keypads
credit card readers (bank-card reader, how many F-keys?)
car park barrier operators
cash machines, etc.
• Strict sequence of operations
• Not much training required
• Useful in public places
Human Computer Interaction 2016, fjv
66
Example Function-Key UI
Human Computer Interaction 2016, fjv
67
Question and Answer Interaction
•
•
•
•
(1) system prompts user, (2) user types answer
Often text-based
Questions asked one at a time
Suitable where short data entry required
– e.g. wizards, installation software
• Limited variation in sequence
– some branching possible
•
•
•
•
Little training needed
Limited support for correcting errors
Slow in use (depending on user)
Control is not with the user
Human Computer Interaction 2016, fjv
68
Form Fill-in
• Includes spreadsheets
• Designed for clerical workers with little
experience
• Interfaces mimic paper forms (metaphor)
• Navigation: designed to move insertion point
without looking at screen
• Structured way of gathering info
• Should be possible to go back
• If transcription: make screen resemble paper
Human Computer Interaction 2016, fjv
69
Form Fill-in
From literature:
Advantages
Disadvantages
• simplifies data entry
• requires modest training
• convenient assistance
(recognition vs. recall)
• allows for formmanagement tools
• consumes screen space
• context and application
dependent
• uses menu style for
parameter fill-in
Human Computer Interaction 2016, fjv
70
Menu Style Interaction
• Covers toolbars, palettes, text-based menus,
etc.
• Selection possible in multiple ways
– pointing
– typing letter or number
– shortcuts instead of pointing
Human Computer Interaction 2016, fjv
71
Characteristics of Menus
• Offer only those actions that make sense in the
current situation (e.g. grey out others)
• Use based on recognition not remembering
• Item names and icon must be carefully chosen
• Not suitable for complex actions
– e.g. commands with many operands
– copying a set of files from one disk to another
• Can be cumbersome way accessing info
– e.g. if wading through a number of submenus is required
• Menu may take up screen space, solution:
– pull-down, pop-up, radial, fish-eye
Human Computer Interaction 2016, fjv
72
Styles of Menus
• There are many variations
–
–
–
–
–
–
–
pull-down & pop-up
command
cascading menus
iconic menus
tree-structured menus (site maps)
pie-menus (pop-up radial)
multiple-column menus (common on WWW)
• Lots of studies have been done
–
–
–
–
how many items/to what depth
dynamically rearranging content according to frequency of use
role of stable position within a pull-down menu
titling (where is the title placed)
Human Computer Interaction 2016, fjv
73
User Feedback with Menus
• Well-designed menu incorporates feedback
• What options are available? (greyed out)
• Visual information
–
–
–
–
–
–
highlighting current item
underlining accelerator keys
‘ticks’ in front of selected options (sound feedback)
‘walk-through’ arrows indicating submenus
dots ‘...’ showing that selection refers to further dialogue
separator lines to group
• Feedback on end of selection process (visibility)
– the appropriate action has occurred
Human Computer Interaction 2016, fjv
74
Example Menu Style
• What options are available?
• ImageJ: java based image processing
Human Computer Interaction 2016, fjv
75
Issues on Single Menu Panel
• Meaningful item name
• Top-level panels of menus most error prone
• It should be easy to go back
– Undo function is useful (incorporated ?)
• IF undo not possible THEN
do not place item at of menu-top
OR near frequently used options
• Critical actions
– do not use accelerator keys
• Try out menus with real users
Human Computer Interaction 2016, fjv
76
Examples Menus (i)
Human Computer Interaction 2016, fjv
77
Examples Menus (ii)
• To see updates from friends, photos and feeds, the Zumobi Ziibii interface
(http://www.zumobi.com) allows users to choose between two styles of presentation.
• On the left is a static list of text/image items with a gestural swipe used to control
paging, and on the right is a dynamic scrolling ticker (called “River”) which
horizontally scrolls titles and images across the screen.
Remark: Newer interaction modes are probed with Menu-structures.
Human Computer Interaction 2016, fjv
78
Organizing a Single Menu Panel
• No ‘right’ number of items
– certainly not 7 ± 2
• Grouping strategy should be applied
– categorical
– conventional
– frequency of use
– importance
– sequence of use
– alphabetical (if all other fails...)
• FishEye menu style (cf. direct manipulation)
Human Computer Interaction 2016, fjv
79
Menu Style :Topics of Concern
1. Naming of items on option list
2. Depth vs. breadth
3. Categorization
4. Time to selection, cf. Fitt’s law:
T = k log2 (D/S+0.5)
T = time to move hand to target
k  100ms,
D = distance,
S = size of target
• How would this work for newer interactions ?
Human Computer Interaction 2016, fjv
80
Menu Selection
From literature:
Advantages
•
•
•
•
Disadvantages
reduces learning time
reduces keystrokes
structures decision making
easy to support error
handling
• too many menus for
complex tasks
• can be slow for frequent
users
• consumes screen space
• requires rapid display rate
Human Computer Interaction 2016, fjv
81
Direct Manipulation
• Windows, Icons, Menus, Pointers, aka WIMP,
Forms, (other objects)
• Objects can be acted upon directly by user with a
pointing device
• Choice of forms and icons important
– a. match:
designer’s representation - user’s understanding
– b. choice of metaphors
– c. explicit attention to ways in which instantiated and ‘real’
objects differ
– d. cultural bias of representation
Human Computer Interaction 2016, fjv
82
Archetypal Direct Manipulation
Human Computer Interaction 2016, fjv
83
Menus in Direct manipulation
Menu Bar (top of screen) menu drags down
• pull-down menu - mouse hold and drag down menu
• drop-down menu - mouse click reveals menu
• fall-down menus
Contextual menu appears where you are
• pop-up menus - actions for selected object
• pie menus - arranged in a circle
• compass menus
84
Direct-Manipulation Systems:
WYSIWYG word processing
Human Computer Interaction 2016, fjv
85
Example Direct Manipulation
Human Computer Interaction 2016, fjv
86
Example Fish-Eye menus
See paper hci.liacs.nl
Human Computer Interaction 2016, fjv
87
Example Fish-Eye Menus
Human Computer Interaction 2016, fjv
88
Review #6
• Interaction Styles
– Command Line
– Natural Language
– Speech
– Key-Modal
– Menus
– Form fill-in
– Direct Manipulation
• Hybrid Styles
– Lecture 7
Human Computer Interaction 2016, fjv
89