CSCI 4620/8626
Computer Graphics
Interactive Input Methods and
Graphical User Interfaces (Chapter 20)
Last update: 2014-03-16
Graphical Input Data
Graphics software can use a variety of input data:
Coordinate positions
Character-string specifications
Geometric transformation values
Viewing conditions
Illumination parameters
Many graphics systems and various standards organizations
(ISO, ANSI) provide input functions for such data.
Input functions are often classified by the type of data they
process.
2
1
Logical Classification of Input Devices
When input functions are classified by data type,
any device that can supply needed data is called a
logical input device for that data type.
Logical input-data classifications:
Locator – specify one coordinate position
Stroke – specify a set of coordinate positions
String – specify some text input
Valuator – specify a scalar value
Choice – select a menu option
Pick – select a component of a picture
3
Locator Devices
Common approach is to use the screen cursor at
some location.
Mouse, touchpad, thumbwheel, dial, touch-screen,
etc. can be used for screen cursor positioning.
Keyboards, particularly arrow keys (with modifiers)
can be used to indicate screen cursor movement.
Light pens can also be used, but there are less
common than the other techniques.
4
2
Stroke Devices
Locator devices can also be used to provide
multiple coordinate positions.
“Click and drag” actions with a mouse provide
stroke input, specifically a start and end coordinate
(as for a line).
Buttons (e.g. on a keyboard) can be used to
modify the actions. For example, pressing a shift
key may constrain coordinate positions to a
horizontal or vertical line.
5
String Devices
The obvious choice for string input is a keyboard.
The keyboard need not necessarily be a “full”
keyboard as found on a traditional computer; it is
often also one of these:
A keyboard with a limited set of keys appropriate to a
specific application
A simulated keyboard displayed on a touch-sensitive
screen
6
3
Valuator Devices
Numerous devices can be used as valuators:
Dials (e.g. potentiometers or rotary switches)
Keyboards
Joysticks, pressure-sensitive devices, etc.
Various on-screen widgets like slider bars, buttons,
rotating scales, etc.
In all cases it is appropriate to provide user
feedback so verification (and correction) of the
input value is possible.
7
Choice Devices
Numerous possibilities exist:
One of many pushbuttons
Keyboards
On-screen menus and cursor-positioning devices
Multi-level menus for complicated choices
Voice input (text suggests this for small # of choices)
8
4
Pick Devices
Can be used to select an entire object, a facet of a surface,
a polygon edge, a vertex, etc.
Using the mouse, we translate a screen coordinate position
to a world coordinate position using inverse
transformations.
Object selection first tries to determine if the coordinate
position is uniquely associated with a single object.
If that fails, then coordinate extents of individual object
components (e.g. line segments) can be tested.
9
Object Selection
If coordinate-extent tests don’t resolve to a single object,
then distance to individual line segments could be
computed (see next slide).
Pick actions can also specify selection of multiple objects at
once, especially if these multiple objects are in a specified
region, or are somehow grouped to represent a single
object.
Keyboards can be used to allow selection of an on-screen
objects and sequencing through the objects.
Named objects can also be selected by keyboard.
10
5
Figure 20-1 Distances to line
segments from a pick position.
Figure 20-2 A pick window with center
coordinates (xp, yp) , width w, and height h
.
6
Graphical Data Input Functions
Typical input options
Interaction mode:
• Program-initiated input
• User-initiated input
• Simultaneous operation
Physical input device selection
When and where to obtain input for a particular data set
13
Input Modes
Request Mode
Application initiates data input
Processing waits (blocked) until input is available
Sample Mode
Application and input devices operate simultaneously
When application needs new data, it “samples” the data that has
been provided
Event Mode
Application and input devices operate simultaneously
Input actions are saved in an event queue, which is processed by
the application when desired
14
7
Echo Feedback
As mentioned earlier, it is appropriate in many
cases for input activity to result in some feedback
to a user.
The feedback can take many forms, depending on
the type of input
Echo of keyboard input
Display of a selection window
Highlighting of selected objects
Range of values or resolution (for valuator input)
15
Callback Functions
These include functions that specify what actions
are to take place when certain events (like input
actions) occur.
Examples include
“normal” and special keyboard entry
mouse button presses and mouse motion
16
8
Interactive Picture-Construction [1]
Basic Positioning Methods
Obvious possibility include using a mouse or a trackball
Specific object type could be specified (e.g. square,
ellipse) and “filled in” given parameters (e.g. from
keyboard or mouse)
Numeric position information could be displayed as
text on the screen (or other device)
Keyboard, dials, etc. could be used to make small
interactive adjustments in coordinate values
17
Interactive Picture-Construction [2]
Dragging
Common technique – select an object and draw to a different
location
Often performed with mouse – press button to select object, move
mouse to desired location, release button
Constraints
With additional input (e.g. from keyboard), the parameters for an
interactively-drawn object can be limited.
For example, holding shift key while drawing a line may constrain
the angle between the line and a coordinate axis
18
9
Figure 20-3 Horizontal line constraint.
Figure 20-4 Vertical line constraint.
10
Grid Constraints
Another technique for constraining vertex positions
is to superimpose an “invisible” grid (with userspecified spacing) over the drawing area.
Now object vertices can be constrained to be on
the intersection of a horizontal and vertical grid
line.
This makes it easier, for example, to guarantee
two line segments share a common endpoint
vertex.
21
Figure 20-5
Construction of a line
segment with endpoints
constrained to grid
intersection positions.
11
Rubber-Band Methods
Used for interactive adjusting object sizes.
For example, one endpoint of a line can be
selected.
Then as the cursor moves, the “draft line” is
displayed between the start position and the cursor
position.
Finally, the end position is selected (e.g. a mouse
button is released).
23
Figure 20-6 A rubber-band method
for constructing and positioning a
straight-line segment.
12
Figure 20-7 A rubber-band
method for constructing a
rectangle.
Figure 20-8 Constructing a
circle using a rubber-band
method.
13
Gravity Field
To identify (or select) positions on objects that might not
match well-defined positions (like grid points), we might
enable an invisible “gravity field” around the objects.
A gravity field is just the region containing all coordinate
positions within a specified maximum distance of an object.
Then we we want to connect a line (for example) with an
object, we only have to select an endpoint that is within the
gravity field of the object; the software automatically
connects the line to the object.
27
Figure 20-9 A gravity field around a
line. Any selected point in the shaded
area is shifted to a position on the
line.
14
Interactive Painting, Drawing
Lines, polygons, circles – use previous methods
Curves – circular arcs, splines, freehand
Splines – specify a set of control points
Line widths, styles, colors and other attribute
options are common
Specification of surface textures, lighting, etc. are
found in more sophisticated systems
29
Virtual-Reality Environments
Interactive input provided with a “data glove”
User appears to be grasping and moving objects displayed in a
virtual scene
The scene is displayed using a head-mounted viewing
system as a stereographic projection; headset orientation
dynamically controls the viewing position
Scene can also be displayed on a raster monitor by rapidly
switching between two stereographic views and switching
the lenses of the user’s glasses on an off in synchronization
30
15
Open-GL Interactive Input Functions
OpenGL programs use GLUT routines to interface
with a window system (as provided by a specific
operating system’s user interface).
Functions are provided for
Input from standard devices (mouse, keyboard)
Other input devices (tablets, space ball, button box, dial)
For each desired input device, a callback function
is specified.
31
Callback Functions
Many graphics packages provide for the
specification of callback functions to handle input
from interactive devices.
If a callback isn’t supplied for a particular device,
then it isn’t used for input by an application.
The callback function is invoked only when the
specified device is “activated.” For example, using
the glut toolkit, glutMouseFunc can be used to
specify the callback function to handle mouse
clicks.
32
16
Common Input Modes
Request Mode – This is the traditional mode used
by many programs (e.g. cin >> x). The input
device(s) and the application operate alternately.
Sample Mode – When the application needs a data
value, it samples the input device (e.g. a
temperature sensor).
Event Mode – Traditional mode for most GUI
(window-based) applications. Input is placed in an
event queue from which items are removed by the
application.
33
Echo Feedback
As a result of an input operation, many interactive
graphic applications will display data and operating
parameters.
This information is usually displayed in a welldefined area of the screen.
Similar applications should ideally use similar styles
and areas for the echo feedback. For example,
compare the placement of menu choices and
parameters for the Microsoft Office programs.
34
17
GLUT Mouse Button Function
glutMouseFunc(mouseFcn);
This call specifies that mouseFcn is the callback function to be
invoked when a mouse button is clicked or released.
void mouseFcn (int button, int motion, int x, int y)
This is the callback function, and has four parameters:
button – which button was involved
• GLUT_LEFT_BUTTON
• GLUT_RIGHT_BUTTON
• GLUT_MIDDLE_BUTTON
motion – GLUT_DOWN or GLUT_UP
x, y – mouse screen position when action occurred
35
GLUT Mouse Button+Movement Function
glutMotionFunc(mmFcn);
This call specifies that mmFcn is the callback function to be invoked
when a mouse is moved while a button is pressed.
void mmFcn (int x, int y)
x, y – mouse screen position (see next slide)
Note that this function will be called multiple times as the
mouse is moved, with differing x and y values, until all
mouse buttons are released.
The particular x and y values, or frequency with which the
function is called, depends on the host (e.g. the operating
system’s) windowing system.
36
18
GLUT Passive Mouse Movement
Function
glutPassiveMotionFunc(pmmFcn);
This function is similar to the “motion+button” function, but
it is only invoked when the mouse is moved without any
button being held down.
Parameters are as for mmFcn (that is, the x and y position
of the mouse pointer).
For all of the mouse functions, the x and y parameters are
the window position of the mouse pointer, with the topleft corner being (0,0).
37
GLUT “Normal” Keyboard Input
glutKeyboardFunc(keyFunc);
This specifies the keyFunc callback to handle “normal”
keyboard input (that is, letters, digits, punctuation that
has a standard ASCII code)
void keyFunc(unsigned char code, int x, int y)
code – the ASCII code for the key that was pressed
x,y – mouse position at the time the key was pressed
38
19
GLUT “Special” Keyboard Input
glutSpecialFunc(skeyFunc);
This specifies the skeyFunc callback to handle “special” keyboard
input – for example, arrow keys, page up/down, F1, F2
void skeyFunc(int code, int x, int y)
code – a GLUT code for the key that was pressed
x,y – mouse position at the time the key was pressed
The values for “code” are GLUT symbolic constants – for
example, GLUT_KEY_UP, GLUT_KEY_PAGE_UP,
GLUT_KEY_F1; see the GLUT specs for complete list.
39
The glutGetModifiers Function
During a mouse or keyboard callback, the function
glutGetModifiers can be called. This function has
no parameters and returns an int.
The result is the bitwise OR of zero or more of
these constants:
GLUT_ACTIVE_SHIFT – if a shift key or the caps lock
key was active at the time the mouse button or
keyboard key was pressed
GLUT_ACTIVE_CTRL – if the control key was active
GLUT_ACTIVE_ALT – if the alt key was active
40
20
Other Input Devices
GLUT includes functions for obtaining input from
various other devices:
Tablets (similar to touch-sensitive input devices, but with
buttons like a mouse)
Spaceballs (don’t move, but senses applied pressure in
various directions – provides 3 dimensional input, and
buttons)
Button boxes
Dials
We won’t cover the input callback functions for
these devices.
41
Table 20-1 Summary of OpenGL
Input Functions
21
Table 20-1 (continued) Summary of
OpenGL Input Functions
Table 20-1 (continued) Summary
of OpenGL Input Functions
22
OpenGL Pick Operations [1]
OpenGL provides functions that can be used to
interactively select objects –to “pick” them.
The basic procedures needed to use these pick
operations are as follows:
Use a designated “pick window” to form a revised view
volume.
Assign integer identifiers to the objects in a scene.
Obtain the identifiers for objects that intersect the
revised view volume from a pick-buffer array.
These operations take place within the mouse
callback function.
45
OpenGL Pick Operations [2]
Objects that are selected are identified in a record
stored in the pick buffer.
Each record contains
The stack position of the object (the integer names of
objects are stored on a stack)
The minimum and maximum depth of the selected
object
The list of identifiers in the name stack from the first
identifier to the identifier for the picked object
46
23
Table 20-1 (continued) Summary of
OpenGL Pick-Related Functions
GLUT Menu Functions (Review)
menuID = glutCreateMenu(menuFcn)
Creates a pop-up menu and identifies the callback
function that handles menu entry selection
“menuID” identifies the menu for use in other functions
void menuFcn(int menuNumber)
Invoked when a menu entry is selected. The single
argument is an integer corresponding to the position of
a selected menu entry.
The entry number (menuNumber) is then used to
perform an operation.
48
24
Adding Menu Entries
One a menu has been created, we can add entries
to it:
glutAddMenuEntry(char *text, int itemNum)
“text” is the text that will appear in the menu, and
“itemNum” indicates where the item will appear in
the menu.
49
Activating a Menu
To active a pop-up menu, we must specify the
mouse button that will cause it to “pop up”:
glutAttachMenu(button);
Here “button” is one of the GLUT symbolic
constants for a mouse button – e.g.
GLUT_RIGHT_BUTTON.
This button is also used to select one of the
menu’s entries.
50
25
Managing Multiple Menus
glutSetMenu(menuID);
Activate menu menuID for the current window.
glutDestroyMenu(menuID);
Eliminates the specified menu.
currentMenuID = glutGetMenu();
Returns the ID of the menu associated with the current
display window.
51
Submenus (1)
As suggested by the name, a submenu is a menu
that is activated by selecting an entry from another
menu.
Using GLUT, a hierarchy of menus can be created.
A submenu itself is prepared in the same way as
another other menu:
Invoke something like this
menuID = glutCreateMenu(f)
to create the submenu; recall that menuID identifies it
Invoke
glutAddMenuEntry(menu text, code)
to add one or more entries to the (sub)menu
52
26
Submenus (2)
The submenu itself is added to the next higher
level directory:
glutAddSubMenu(menu text, submenuID)
submenuID is the integer identification returned when
the submenu was created
Example menu programs named menu.cpp and
submenu.cpp are available on loki.
53
Modifying Menu Items
The button used to activate a menu and select a
menu item can be changed.
First, cancel the current button assignment:
glutDetachMenu(mouseButton);
mouseButton is something like GLUT_RIGHT_BUTTON
Now use this call to reattach the menu:
glutAttachMenu(differentmouseButton);
A menu entry can be deleted like this:
glutRemoveMenuItem(itemNumber)
itemNumber is the integer value of the menu item
to be deleted.
54
27
Table 20-2 Summary of OpenGL
Menu Functions
Designing a Graphical User Interface
Graphical User Interfaces (GUIs) contain numerous
graphical elements (display windows, icons,
menus, etc.) used to facilitate user interaction with
an application.
GUIs for various applications often employ similar
features allowing users familiar with one
application to perform similar tasks using other
applications.
GUI design and human-computer interaction is a
large and complex area, which we obviously won’t
cover in much detail here.
56
28
GUIs: The User Dialogue
The dialogue, or interaction with the user, can be
tightly controlled, but more frequently the order of
performing tasks is flexible.
The language used in the dialogue is selected to
be appropriate to the application area.
The application area also indicates the kinds of
“objects” the user will be able to create and
manipulate.
57
GUIs: Windows and Icons
One or more windows, as appropriate to the
application, are provided by a GUI.
In addition to traditional window operations,
additional “widgets” (sliders, buttons, icons, etc.)
are used to provide input to the application.
Some icons may represent objects (e.g. walls in an
architectural application, or capacitors in the circuit
drawing application).
Other icons may represent various action
possibilities, including traditional actions like cut
and paste.
58
29
GUIs: Accommodating Multiple Skill
Levels
Many GUIs attempt to facilitate use by users with varying skill levels.
Normally, a minimal set of operations and features are exposed by the
GUI.
As a user becomes more experienced, they may enable additional
features, which may result in a larger number of menu choices, icons,
etc. Many of these features may also be provided with keyboard
shortcuts (like control-C for copy and control-V for paste in MS
Windows).
Users may often be able to tailor “tool bars” and other menus to
provide those features they use most frequently; this can improve the
speed with which the user can use the application.
Help facilities can also have multiple levels, and tutorials are usually
provided.
59
GUIs: Consistency
As noted earlier, consistency in the method of
invoking common operations (e.g. cut and paste)
among a large number of applications is expected.
Additionally, consistency within an application is
important, too. If a particular operation is
performed by clicking a triangular icon in one
application window, then the same icon shape
should be used in other windows for the same
application.
Consistency also applies to things like color
schemes, font selection, and so forth.
60
30
GUIs: Minimizing Memorization
Given that some application may provide hundreds
or even thousands of features, it’s normally
impossible for a user to memorize all of them.
Instead, features should be grouped appropriately
so, for example, all editing features in a word
processor can be found in the same general area.
“Hover text” – messages that appear when a
pointing device “hovers” over an icon – provide a
convenient way of reminding a user of an icon’s
purpose.
61
GUIs: Backup
Almost all GUIs attempt to provide features to
“undo” a sequence of operations.
In many word processing applications, for
example, pressing control-Z will “undo” the last
editing operation, even if it affected a large piece
of text.
Most applications even allow “undo” to extend
backward over a relatively large number of actions.
Operations that cannot easily be “undone” should
also be accompanied by dialogues requesting user
confirmation: e.g. “quit without saving?”
62
31
GUIs: Error Handling
It’s also important for a user interface to inform a
user of errors that occur, and perhaps provide
rational approaches to deal with them.
Errors may also be anticipated, and user
confirmation solicited before performing operations
that may result in an error.
63
GUIs: Feedback
As always, providing clear, consistent feedback on
an applications behavior is important.
Each user action should cause some feedback:
Highlighting a selected object, text string, etc.
Displaying an icon or a message
Displaying an icon as “grayed” if its unavailable
Showing an hourglass or other cursor shape to suggest
the application is busy
Feedback should be clear, but not overpowering.
Too much feedback can be as bad as too little:
trees get lost in the forest.
64
32