PLATEAU 2010
Toward Transforming Freely
Available Source Code into Usable
Learning Materials for End-Users
Paul Gross and Caitlin Kelleher
Washington University in St. Louis
PLATEAU 2010
Illustrating Available, Unusable Code
Call
your relative
Find/Select
working on a Ph.D. in
Relevant
Code
Computer
Science
Use Relevant
Code
2
PLATEAU 2010
An Independent Learning Process for
End-user Programmers
Search the
Search the
Web for
Web for
Examples
Examples
[1,2]
Evaluate
Example
Relevance
Select
Relevant
Code
Use in
Use in
Personal
Personal
Context
Context
[1,2]
[1] Brandt et al., Two studies of opportunistic programming: interleaving web…, CHI 2009.
[2] Rosson et al., Who, what, and how: A survey of informal and professional…, VL/HCC 2005.
PLATEAU 2010
Our Context: Looking Glass
PLATEAU 2010
An Independent Learning Process for
End-user Programmers
Search the
Web for
Examples
Evaluate
Example
Relevance
Select
Relevant
Code
Use in
Personal
Context
PLATEAU 2010
Evaluation by Inexperienced End-Users
• Inexperienced end-users envision output
PLATEAU 2010
Evaluation by Inexperienced End-Users
Executable Program
Graphical Output
Relevance Decision
PLATEAU 2010
Example Code Usability for
Inexperienced End-Users
Evaluation
Example
Programs/
Snippets
Does this exhibit
a feature I am
looking for?
Read code,
descriptions,
execute
program
Selection
Relevant
Example
Where is the code
corresponding to the
feature I want?
PLATEAU 2010
Non-Programmers’ Selection
Search Model
Internal Landmark Model
Mappings
Output
Landmark Set
Code
Landmark Set
9
[3] Gross and Kelleher. Non-programmers identifying functionality…, JVLC (21), 5, Aug 2010.
PLATEAU 2010
Mapping Code to Output: Debuggers
PLATEAU 2010
Hastings: an Output History Explorer Tool
PLATEAU 2010
Do these Tools
Adequately Assist
Non-Programmers
with Selection?
PLATEAU 2010
Evaluative Study:
Hastings vs. a Debugger
PLATEAU 2010
Looking Glass Debugger Threads
Looking Glass Program Code
Thread Presentation
PLATEAU 2010
Evaluative Study:
Hastings vs. a Debugger
18 NonProgrammers
9 Hastings
Users
9 Debugger
Users
PLATEAU 2010
Evaluative Study: Tasks
• Find and mark code corresponding to
graphical output
PLATEAU 2010
Do these Tools Adequately Assist NonProgrammers with Selection?
Quantitative Performance
Qualitative Barriers
PLATEAU 2010
Quantitative Performance
Average Tasks
Completed
p < .01
3.33
3.5
Percentage of
Correct Tasks
p < .05
72.8%
80%
3
2
1.5
1
40.2%
1.22
40%
20%
12:00
9:36
7:12
4:48
2:24
0.5
0
16:48
15:07
14:24
60%
2.5
Average Time on
Task
p = .06
0%
Debugger
0:00
Hastings
8:58
PLATEAU 2010
Qualitative Barriers
Debugger Barriers
• Thread Misunderstanding
• Step Over Misinterpretation
Hastings Barriers
• Concurrency Ambiguity
• Playback Fidelity
PLATEAU 2010
A Qualitative Barrier
Concurrency Ambiguity [Hastings]
PLATEAU 2010
Do these Tools
Adequately Assist NonProgrammers with
Selection?
Room for improvement
PLATEAU 2010
Future Design Guidelines
Do not rely on execution
abstractions
Provide affordances for
replaying code elements
Use direct code interactions
PLATEAU 2010
An Independent Learning Process for
End-user Programmers
Search the
Web for
Examples
Evaluate
Example
Relevance
Select
Relevant
Code
Use in
Personal
Context
PLATEAU 2010
Further Challenges
Search the
Web for
Examples
Evaluate
Example
Relevance
Select
Relevant
Code
Use in
Personal
Context
• Searching with inexperienced end-users’
evaluative vocabulary
PLATEAU 2010
Further Challenges
Search the
Web for
Examples
Evaluate
Example
Relevance
Select
Relevant
Code
Use in
Personal
Context
• Supporting efficient evaluation of many
search results
• Enabling snippet evaluation
PLATEAU 2010
Further Challenges
Search the
Web for
Examples
Evaluate
Example
Relevance
Select
Relevant
Code
Use in
Personal
Context
• Supporting complex selection of code
areas
PLATEAU 2010
Further Challenges
Search the
Web for
Examples
Evaluate
Example
Relevance
Select
Relevant
Code
Use in
Personal
Context
• Promoting learning from personally
motivating examples
• Assisting with reuse and adaptation
PLATEAU 2010
Questions?
• {grosspa, ckelleher}@cse.wustl.edu
• http://lookingglass.wustl.edu
References
• [1] J. Brandt, P. J. Guo, J. Lewenstein, M. Dontcheva, and S. R. Klemmer.
Two studies of opportunistic programming: interleaving web foraging,
learning, and writing code. In Proc. of CHI, pages 1589–1598, 2009.
• [2] M. B. Rosson, J. Ballin, and J. Rode. Who, what, and how: A survey of
informal and professional web developers. In Proc. of VL/HCC, pages
199–206, 2005.
• [3] P. Gross and C. Kelleher. Non-programmers identifying functionality in
unfamiliar code: strategies and barriers. JVLC, 21 (5):263–276, Aug. 2010.