CAN BLIND PROGRAMMERS PROVIDE
(ACCIDENTAL) INSIGHT INTO
PROGRAMMING LANGUAGE DESIGN?
Andreas Stefik, Ph.D.
Assistant Professor
Computer Science
Southern Illinois University Edwardsville
BLIND INDIVIDUALS FACE
UNIQUE CHALLENGES WHEN
LEARNING TO PROGRAM
Integrated development environments are
often poorly accessible
NetBeans
XCode
Visualization is (obviously) not a sensible
option for Blind Individuals
Alice
Unity 3D
Given that programming languages use text, they are a
viable option for blind students
How well do
programming
languages “read”
through a screen
reader?
SYNTAX IS HARDER TO
UNDERSTAND THROUGH AUDIO
C-style syntax is hard to understand
using audio
for(int i = 0; i < 10; i++) {
System.out.println(i);
}
This translates to:
for int i equals zero semicolon i less
than ten semicolon i plus plus right
parent left brace
Initially, we thought a new language might be
easier to “read” with a screen reader
integer i = 0
repeat 10 times
print I
i=i+1
end
1) Use plain English
2) minimize the use of
esoteric symbols (e.g.,
{}, ||, &&, ===)
3) Be terse, but clear
We also used statistical measures of auditory
comprehension to design talking debuggers
Loop Iteration 1
Loop Iteration 2
1 Nested If True
Talking Debugger
1 Nested If False
End Loop
Artifact Encoding
Measure
Scoping
p-val
.001∗∗
Comprehension
.051
Missing if
.451
Missing loop
.102
Flipped boolean
.144
Scoping Cues
Yes
No
M = 79.8
M = 59.2
S D = 13.1 S D = 10.1
M = 62.53 M = 52.26
S D = 25
S D = 16.8
M = 25.13 M = 18.97
S D = 29.9 S D = 31.5
M = 8.81
M = 5.16
S D = 9.7
S D = 8.35
M = 7.6
M = 5.4
S D = 8.1
S D = 4.7
p
.
.
.
.
.
Table 3: A summary table of the experimental results (∗ = α < 0.05, ∗∗ = α < .01). The sco
with a maximum value of 100%, whereas the missing if, loop, and flipped booleans measures a
2011 Java Innovation Award
OUR SCREEN READER FRIENDLY
PROGRAMMING LANGUAGE “SEEMED”
EASIER TO UNDERSTAND IN GENERAL
Before starting the design of Quorum, we
thought, let’s ask novices what they think
10
8
Non−programmer
●
6
●
●
●
●
●
●
●
●
●
4
●
●
●
0
10
8
●
●
●
●
●
●
●
4
●
●
reoccur
●
redo
6
●
●
●
●
2
while
rerun
repeat
loop
iterate
foreach
for
echo
duplicate
do
cycle
again
0
Programmer
Intuitiveness
2
●
The words for,
while, and
foreach make
no sense.
Our broad goal of these surveys was to find out, “What
words/symbols do novices think we should use in a
programming language?”
Functions with return values
10
8
8
●
●
4
●
●
4
8
●
●
Programmer
●
●
●
4
C++
Go
Java
Perl
PHP
Python
Quorum
Ruby
Smalltalk
●
●
●
●
2
0
10
●
●
●
6
●
●
●
●
●
●
●
●
●
●
●
●
●
8
●
●
●
●
●
6
●
●
●
●
●
●
●
4
●
2
2
0
0
●
●
●
Exceptions/Throw
10
8
●
●
●
●
●
●
●
●
4
●
●
●
●
●
●
●
Non−programmer
●
6
2
Intuitiveness
●
●
●
0
10
8
6
●
●
●
●
●
●
●
4
●
●
●
●
●
●
●
●
●
2
toggle
throw
signal
repair
raise
manage
hold
handler
handle
fix
error
do
correct
call
alert
alarm
0
Programmer
We asked hundreds of novices what
they thought the syntax of a
programming language “should” be.
●
●
4
0
10
8
●
●
zero
without
void
vacant
unused
undefined
selected
null
nothing
none
nil
empty
defined
blank
bare
0
●
●
this
source
self
seed
residential
pointer
parent
offspring
myself
me
home
here
destination
constant
child
●
2
●
●
●
●
2
4
●
●
●
6
●
●
●
●
6
Programmer
Intuitiveness
0
10
●
●
●
Programmer
Intuitiveness
2
●
Intuitiveness
●
●
●
Non−programmer
●
●
●
8
●
Non−programmer
●
10
6
Non−programmer
6
Null
This
10
When we talk to novices about programming
languages, they say …
That’s not
Greek, it’s
Klingon*
Standard Java Syntax:
for(int i = 0; i < 10; i++) {
}
From a news article comparing Quorum to Perl:
http://www.fastcodesign.com/1665735/why-arent-computer-programming-languages-designed-better
EMPIRICAL STUDIES WITH NOVICES
MAY REVEAL IMPROVEMENTS FOR
PROGRAMMING LANGUAGES
We setup a study on novice accuracy rates between
three programming languages
Randomo
From medicine, we
adapted the idea of a
“Placebo” by randomly
selecting syntax from the
ASCII table
Novice Perl users could not program significantly more accurately than
those using a programming language with randomly generated
keywords
1.0
Results show Quorum > (Perl = Randomo)
0.8
0.6
●
●
●
●
0.4
Perl
Quorum: (M=.628, SD=.198)
Perl: (M=.432, SD=.179)
Randomo: (M=.341, SD=.173)
●
●
0.2
0.0
1.0
0.8
●
0.6
●
Quorum
Accuracy
●
●
●
●
0.4
0.2
0.0
1.0
0.8
Perl users performed no
better than those using a
Placebo? Let’s run a
replication.
●
0.4
●
●
●
●
●
0.2
0.0
1
2
3
4
Tasks
5
6
Randomo
0.6
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
Ruby
●
●
●
●
1
2
3
4
Tasks
Randomo
●
●
●
Quorum
●
●
●
●
●
●
Python
●
Perl
1.0
0.8
0.6
0.4
0.2
0.0
1.0
0.8
0.6
0.4
0.2
0.0
1.0
0.8
0.6
0.4
0.2
0.0
1.0
0.8
0.6
0.4
0.2
0.0
1.0
0.8
0.6
0.4
0.2
0.0
1.0
0.8
0.6
0.4
0.2
0.0
Java
Accuracy
Java and Perl users performed no better than Placebo
Users (Ruby, Python, and Quorum users did)
5
6
STY L OS, J. A ND CL A RK E, S. 2007. Usability implications of requiring parameters S
inTEFIK
objects’
constructors.
Software
, A .,
H UNDHAUSEN
, C., AEngiND PATTERSON , R. 2011. An empirical investig
neering, 2007. ICSE 2007. 29th International Conference on, 529–539.
enhance computer program comprehension. International Journal of Human-C
STY L OS, J. A ND M Y ERS, B. A . 2008. The implications of method placement on APISTEFIK
learnability.
SIGSOFT ’ 08/FSE-16:
, A ., HInUNDHAUSEN
, C., A ND SM I TH , D. 2011. On the design of an educ
Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of visually
Softwareimpaired
Engineering.
ACM, New
in computer
science. In Proceedings of The 42nd ACM Tech
York, NY, USA, 105–112.
Education. ACM, New York, NY.
TAY L OR, J. 1990. Analysing novices analysing prolog: what stories do novices tell S
themselves
prolog?
Instructional
TEFIK , A .,about
SI EBERT
, S., S
TEFIK , M ., A ND SL ATTERY, K . 2011. An empirical com
Science 19, 283–309.
using the quorum, perl, and randomo programming languages. In Proceedings
T EI TEL BAUM , T. A ND REPS, T. 1981. Thecornell program synthesizer: a syntax-directedEvaluation
programming
Comandenvironment.
usability of programming
languages and tools. PLATEAU ’ 11.
mun. ACM 24, 9, 563–573.
STEI M A NN , F. 2006. The paradoxical success of aspect-oriented programming. In
V I CK ERS, P. A ND A LTY, J. L . 2002. When bugs sing. Interacting with Computers 14, 6,annual
793–819.
ACM SIGPLAN conference on Object-oriented programming systems, la
New York, NY, 481–497.
STY L OS, J. A ND CL A RK E, S. 2007. Usability implications of requiring parameters
i nt eger . 5, . 42 y . 83, 1 = 1, 1 6547 1, . 92
1, 1
neering, 2007. ICSE 2007. 29th International Conference on, 529–539.
if
1, . 75 y 1, . 83 = 1, . 67 1000 1, . 92
1, . 83 t hen . 67, . 5
STY L OS, J. A ND M Y ERS, B. A . 2008. The implications of method placement on AP
y . 33, . 17 = . 5, . 25 y 1, . 75 + 1, . 75 1000 1, 1
. 83, . 92
Proceedings of the 16th ACM SIGSOFT International Symposium on Foundatio
end . 33, . 08
York, NY, USA, 105–112.
TAY L OR, J. 1990. Analysing novices analysing prolog: what stories do novices tell
el s e . 33, . 5 t hen . 17, . 08
Science 19, 283–309.
if
1, . 5 y 1, . 92 > 1, . 92 1000 1, 1
1, . 92 t hen . 67, . 67
T EI TEL BAUM , T. A ND REPS, T. 1981. Thecornell program synthesizer: a syntax-dir
y . 33, . 17 = . 5, . 25 y 1, . 75 - 1, . 83 1000 1, 1
. 83, . 92
mun. ACM 24, 9, 563–573.
end . 33, . 08
V I CK ERS, P. A ND A LTY, J. L . 2002. When bugs sing. Interacting with Computers 1
Our statistical models can estimate per-token
accuracy rates, leading to changes in Quorum 1.7
Results suggest Quorum should allow:
1. Limited type inference
2. More “Ruby-like” if statements
el s e
. 5, . 42
y
end
end
t hen
. 33, . 17
=
. 5, . 25
. 5, . 25
y
1, . 75
+
1, . 83
2000
. 67, . 5
1, 1
. 83, 1
i nt eger
if
. 5, 0
. 5, . 42
1, . 75
y
y
. 33, . 08
el s e
. 33, . 5
Quorum 1.0 Syntax
if
1, . 5
y
end
=
. 5, . 25
t hen
y
1, . 67
y
1, 1
6547
1000
1, . 75
1, . 92
1, . 92
+
1, 1
1, . 83
1, . 75
1000
1, 1
1, . 92
t hen
1, 1
. 67, . 5
. 83, . 92
. 17, . 08
1, . 92
. 33, . 17
. 5, . 42
y
end
=
=
>
1, . 92
=
. 5, . 25
t hen
. 5, . 25
1000
t hen
. 67, . 67
y
1, . 75
-
1, . 83
1000
1, 1
. 83, . 92
y
1, . 75
+
1, . 83
2000
1, 1
. 83, 1
. 33, . 08
el s e
end
. 83, 1
1, . 83
. 33, . 17
Fig. 15. Token Accuracy Map for Quorum, Taskend6.
y
. 33, . 17
=
. 5, . 25
. 67, . 5
. 5, 0
Fig. 15. Token Accuracy Map for Quorum, Tas
Quorum 1.7 Syntax
One blind student, who has programmed in Python
and Quorum, summed it up by saying:
I have been, you know
looking at it, and ... the
syntax is just very simple to
use. I can just remember
most of the keywords and so
I just think it is pretty nice
and flexible.
From an interview with a blind
student from Tennessee on
Quorum 1.6, Sodbeans 3.0, and
our curriculum/textbook
Working with blind individuals made us re-think the
design of programming languages, which may benefit
everyone
Quorum: http://quorum.sourceforge.net/
Sodbeans: http://sodbeans.sourceforge.net/
Quorum 1.7 and Sodbeans 3.5
Early February