IT 12 009
Examensarbete 30 hp
April 2012
Providing a web view for monitoring
Erlang systems
Nicklas Nordenmark
Institutionen för informationsteknologi
Department of Information Technology
Abstract
Providing a web view for monitoring Erlang systems
Nicklas Nordenmark
Teknisk- naturvetenskaplig fakultet
UTH-enheten
Besöksadress:
Ångströmlaboratoriet
Lägerhyddsvägen 1
Hus 4, Plan 0
Postadress:
Box 536
751 21 Uppsala
Telefon:
018 – 471 30 03
Telefax:
018 – 471 30 00
Hemsida:
http://www.teknat.uu.se/student
Tools for monitoring Erlang nodes exist but they have
not aged well. With the help of new web technologies
such as HTML5, JavaScript and CSS3 more modern
interfaces can be developed. However, to combine an
Erlang backend with a web front-end requires a very
broad skill set. This thesis aims to present a prototype
framework to simplify the construction of web based
interfaces for Erlang developers. The framework
consists of multiple layers to achieve modularity (such
as supporting multiple Erlang web servers).
Functionality is implemented in so called components
(with similarities to an MVC structure) that each
consists of an Erlang module, HTML templates, a CSS
file, a JavaScript file and a config file. While the main
purpose was to implement monitoring functionality,
the framework is general enough to implement custom
functionality.
Evaluation was conducted using usability testing and
this helped to identify problems which led to a change
in the framework structure.
Handledare: Adam Lindberg
Ämnesgranskare: Justin Pearson
Examinator: Anders Berglund
IT 12 009
Tryckt av: Reprocentralen ITC
Populärvetenskaplig sammanfattning
Kompletta datorsystem består idag av ett flertal program och kan i sin tur även
vara uppbyggda av flera mindre datorsystem. De olika programmeringsspråken
ger ofta tillgång till statusinformation om systemets hälsa och belastning. Utöver
statusinformation finns ofta också information om vilka delsystem som körs och
vad de har för uppgift just nu. Denna information presenteras på traditionellt sätt
i textbaserad form och hämtas från ett antal olika platser. Detta gör det svårt att
med traditionella verktyg få en överblick över statusen på större system. Målet
med detta examensarbete är att råda bot på detta genom att föreslå ett modernare webbaserat gränssnitt för att presentera statusinformation om datorsystem
grafiskt.
Utveckling på webben blir allt mer populärt och idag är det möjligt att producera både interaktiva och eleganta gränssnitt som samtidigt är oberoende av operativsystem. I samband med att nya teknologier för utveckling av webbgränssnitt
släpps så ökar dock även kraven på utvecklarna. Fram tills för några år sedan var
det inte sällan samma personer som arbetade både på server-sidan och klientsidan
medan det idag kan vara svårt att hitta personer med så pass bred kompetens.
Det är idag inte ovanligt att olika utvecklare fokuserar på gränssnitt och presentation respektive det underliggande systemet med databas och leverans/hantering av data. Att följa med i de senaste trenderna och framstegen inom webb
(JavaScript/jQuery, HTML5 och CSS3) kan vara svårt att kombinera med en
spetskompetens inom databaser och transportformat (till exempel JSON). Detta
examensarbete syftar till att presentera ett ramverk för att förenkla utvecklingen
av sådana gränssnitt (främst) för användare av programmeringsspråket Erlang. I
teorin kan detta ramverk dock appliceras även på andra programmeringsspråk.
Olika metoder för att utvärdera ramverket undersöktes och i slutändan användes
så kallad usability testing. Denna metod går ut på att låta testpersoner utföra ett
antal givna scenarier på produkten som önskas testas. De scenarier som togs fram
baserades på riktlinjer som var relevanta för utvärderingar av produkter såsom
detta ramverk. Dessa scenarier var värdefulla stöd för testarna och hjälpte till att
identifiera och belysa en del problem som ledde till en förändring av ramverkets
struktur. Tanken var att utföra testning på det här sättet i iterationer men på
grund av tidsbrist så hanns bara en iteration med.
Tack vare utvärderingen så kunde en protyp av ett generellt ramverk tas fram
som stödjer utvecklingen av fristående och mer specifika komponenter. Ramverket
blev väl mottaget hos testarna och har potential att kunna utvecklas till ett system
att monitorera Erlang-system.
3
Contents
1 Background
1.1 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Problem Description . . . . . . . . . . . . . . . . . . . . . . . . . .
8
8
8
2 Methodology
2.1 Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9
9
3 System Overview
12
3.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.2 System Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . 13
4 Implementation
4.1 JSFW Core . . . . . . . .
4.2 JSFW Webserver API . .
4.3 JSFW JavaScript API . .
4.4 JSFW Erlang API . . . .
4.5 Documentation . . . . . .
4.6 JSFW Webserver Plugins .
4.7 JSFW Components . . . .
4.8 Erlang module . . . . . . .
4.9 JavaScript file . . . . . . .
4.10 CSS file . . . . . . . . . .
4.11 Constructing a Component
5 Evaluation
5.1 Evaluation Methods .
5.2 Evaluating JSFW . . .
5.3 Results . . . . . . . . .
5.4 Resolution of Problems
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
13
14
14
15
16
16
17
17
17
18
19
19
.
.
.
.
20
21
21
23
24
6 Conclusion and discussion
25
6.1 Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
7 Future Work
7.1 WebSockets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.2 Support for Additional Webservers . . . . . . . . . . . . . . . . . .
7.3 Integration of a template engine . . . . . . . . . . . . . . . . . . . .
26
26
26
26
8 References
27
5
A Scenarios
29
A.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
A.2 Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
A.3 Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
B Questions
C Framework Documenation
C.1 Structure . . . . . . . .
C.2 URL Scheme . . . . . .
C.3 Component structure . .
C.4 JavaScript API . . . . .
C.5 Erlang API . . . . . . .
30
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
31
31
32
32
35
35
D Source Code
36
D.1 Cowboy Web server plugin . . . . . . . . . . . . . . . . . . . . . . . 36
D.2 Misultin Web server plugin . . . . . . . . . . . . . . . . . . . . . . . 37
D.3 Example Components . . . . . . . . . . . . . . . . . . . . . . . . . . 38
6
1
Background
As systems grow more and more complex the need for easy-to-use monitoring tools
increases. The size and number of components in a system grow and gaining an
overview becomes more difficult. Traditionally, Erlang nodes are usually monitored via the console. This means querying the system using Erlang functions or
parsing log files to diagnose the system. Graphical monitoring applications (e.g.
WebTool [1] and Appmon [2]) are available but their interfaces have not aged well
and most will not work remotely. The maturing front end technologies including
HTML5, CSS3 and JavaScript are becoming increasingly popular as they provide
a multitude of features that allow developers to build interactive and attractive
interfaces.
1.1
Related work
Functionality wise, the most relevant related works are Appmon, WebTool and
BigWig. Appmon (short for application monitor) is a tool that allows users to
monitor processes running on the local machine. Process information, such as
memory usage, is displayed in a graphical interface. WebTool allows users to start
web based Erlang tools such as a web version of Appmon. WebTool also acts as
a framework for developing new web based tools. Tools developed using WebTool
simply output raw HTML code from Erlang modules. This thesis takes some of
its inspiration from an entry in the Spawnfest competition [3] called BigWig [4].
It is a web-based monitoring tool for Erlang nodes where the user can get basic
information about the node it runs on. The information presented is accessible
through the Erlang shell but is here presented in a much more user-friendly way,
without the need for scrolling through terminal output in order to find what is
being sought after. The ambition of BigWig was to supersede monitoring tools
such as WebTool [1] and Appmon [2].
1.2
Problem Description
As front-end development becomes more and more sophisticated, the need for
dedicated interface developers increases. In many cases it gets harder to implement
front end systems with an existing team since they need a knowledge of modern
technologies in both the front end and back end. This paper presents a prototype
implementation of a library that will aid the Erlang developer seeking to develop
fairly advanced web based graphical user interfaces where back-end data is to be
presented. The ambition is to allow these developers to present the same type of
information as Appmon does but in a web based interface. This is essentially what
WebTool does but here we try to take this further by utilizing modern technologies
7
such as HTML5, jQuery, CSS3 and JSON. This is a somewhat apt description of
the BigWig project. However, unlike the BigWig project, we aim to produce a
framework that is so general that people can develop their own custom components
to suit their needs.
2
Methodology
This project was mostly conducted in an iterative manner with the development
of individual components as foundation. The initial idea was to try to focus on
the realization of components similar to those prototyped by the BigWig project,
since they cover most basic monitoring setups.
2.1
2.1.1
Tools
Erlang
Erlang is a general-purpose programming language and runtime environment developed by Ericsson. It has built-in support for concurrency, distribution and
fault tolerance [5, 6]. In 1998 Ericsson decided to release Erlang as open source
and today it is available online1 .
2.1.2
JavaScript
JavaScript2 is a widely used scripting language primarily used in front-end web
development. Lately, JavaScript has also gained ground on the server-side thanks
to frameworks such as Node.js3 .
2.1.3
jQuery
jQuery4 is a JavaScript library that claims to make many JavaScript operations
(e.g. document traversal and Ajax functionality) easier. jQuery was used to allow
changes in the interface layout without reloading the web page.
2.1.4
JSON
There was a need to transform data to a uniform format that could be understood by both Erlang and JavaScript. JSON5 was chosen as this data transport
1
http://www.erlang.org/
https://developer.mozilla.org/en/JavaScript
3
http://nodejs.org/
4
http://jquery.com/
5
http://json.org/
2
8
format since it is so widely used and implementations exist for many di↵erent
programming languages (including Erlang).
2.1.5
JSX
JSX [7] was chosen as the JSON encoding / decoding library since there exists a
forked version of it [8] which includes support for additional Erlang terms such as
process ids (PIDs), ports, functions and tuples. A comparative benchmark suite
for comparing di↵erent Erlang to JSON libraries exists [9] and although JSX is
not the top scorer when looking at the benchmark results [10] we decided to use
it since it supports the Erlang terms mentioned above (that are commonly used
in monitoring systems) as well as being standalone (unlike Mochijson from the
Mochiweb web server [11]).
2.1.6
HTML
HTML use used to describe the basic layout of elements on any website. Apart
from using basic HTML elements for presentation, HTML56 adds many new and
interesting features such as more semantic markup elements (header, footer, article
etc.), client-side validation and a History/State API for manipulating the browser
history.
2.1.7
History.js
History.js7 is a JavaScript library that aims to supersede jQuery History8 . Both
of these JavaScript libraries make it easier to make use of the History/State API
introduced with HTML5 which is advantageous to do when writing asynchronous
web front ends. This was used to make sure that navigation of the browsing history
worked as intended while the website was changed/updated using JavaScript.
2.1.8
CSS
Cascading Style Sheets (CSS)9 is a mechanism for adding custom styling to web
documents. A new version (CSS3) is currently under development10 . A small fraction of styling for common elements (such as tables) was included in the framework.
6
http://www.w3.org/TR/html5/
https://github.com/balupton/History.js/
8
https://github.com/balupton/jquery-history
9
http://www.w3.org/Style/CSS/
10
http://www.w3.org/TR/CSS/
7
9
2.1.9
Web server
A web server listens and responds to requests from clients (usually end users using
browsers). Relevant content (usually HTML documents) is then sent back to
the client. Below the two applications that were used as web servers are briefly
described.
2.1.10
Cowboy
Cowboy11 is described as nothing more than a TCP acceptor pool, forwarding
requests to specified handlers. Included is an HTTP handler so Cowboy can easily
be set up as a web server. This was a target web server early on because of its
web sockets support, which we initially planned on implementing.
2.1.11
Misultin
Misultin12 is an Erlang library for building fast lightweight HTTP servers. For the
same reason as Cowboy, this was also a target web server.
3
System Overview
BigWig is a good system and can manage the monitoring of the most basic sections
(processes, applications, system load, etc.). However, if a user wants to manage
a specific part of his or her system (a list of online users in a chat system for
example), then he or she is at a loss with BigWig. Thus, a system that extends
the functionality of BigWig was to be implemented. Based on this, a list of system requirements was constructed to guide the implementation and help identify
critical features.
3.1
Requirements
The requirements were categorized in the following four categories and by fulfilling these requirements we would have achieved a quite powerful framework for
monitoring Erlang systems.
Components System
First of all, the system has to allow for specific functionality like the case described
above with the list of users. By definition, we call the part that takes care of a
11
12
https://github.com/extend/cowboy
https://github.com/ostinelli/misultin
10
specific functionality a component. This can quite naturally lead to a multitude
of components in a large system and therefore we wanted independence among
components as well as the ability to easily insert or remove individual components
in the system. This also means that the framework itself must come with a set of
externally accessible API functions for the components to use.
User Interaction
Another interesting feature that opens up a lot of di↵erent implementations is the
ability for users to interact with the system. We want the framework to allow
users to send messages to Erlang processes as well as input to Erlang modules.
Integration
Since this is supposed to be a monitoring framework, we want it to integrate easily
with currently running systems that need to be monitored. Since we are exposing
a web interface we need to be able to run on any (within reason) Erlang web
server that the current system might be using. Therefore, we need a web server
abstraction layer and plugins for the di↵erent web servers that can account for
this.
Modern Technology
Just like BigWig, this framework should be built using modern web technologies
including HTML5, CSS and JavaScript/jQuery.
3.2
System Architecture
Based on the requirements above, an abstract sketch of the system architecture
was constructed (see Figure 1).
Figure 1: Flow chart of the JSFW system on an abstract level.
11
4
Implementation
The framework is called JSFW (JavaScript FrameWork). The framework was
developed in multiple layers to support flexibility and modularity. The first layer
is a web server abstraction layer that is supposed to rid the core framework of
any web server dependency. Next layer contains the general functionality of the
framework and takes care of forwarding requests to the correct component in the
final layer. This final layer contains the components which in turn contain specific
isolated functionality. These di↵erent layers are described in more detail below.
To allow for easy integration without database setup, persistent data (such as
component meta data) was stored in configuration files and a list of components
was maintained by searching a path.
4.1
JSFW Core
This is the core system that takes care of HTTP request routing, component and
plugin setup and provides an API for constructing the Webserver plugins and the
components. This part contains both JavaScript and Erlang since it does work
both client-side and server-side. In the core part you can also find a pre-built
component that deals with displaying process information based on PIDs and also
an interface for managing your available components. What is also included is
some pre-defined CSS styling for common elements.
4.2
JSFW Webserver API
This is the web server abstraction layer where HTTP requests are translated to
a common format (implemented as an Erlang record, #jsfw http). This request
then propagates to JSFW core where it is handled and a reply is constructed
(#jsfw reply). This reply is translated to a format that is understood by the
corresponding web server and then returned to the client via the web server. The
motivation for this translation was to enable support for multiple web servers.
4.2.1
#jsfw http
This record is filled with selected information extracted from the HTTP request
coming from the web server. Only information that is required by JSFW core is
extracted to avoid information duplication.
• The request method13 (only GET and POST are allowed at this stage).
13
http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html
12
• The request path to allow us to respond to the request based on what resource
is being requested.
• Request query (the part after the ? in URLs) to include arguments with the
request. This can be used to specify in detail what information is requested
(for example details=name,age).
An example record of this type could look something like this:
#jsfw_http{
method="GET",
path="/users/1",
query="details=name,age"
}
4.2.2
#jsfw reply
The record holding the reply information is, just like the request record, straight
forward and contains the following information.
• An HTTP status code14 to tell the web server how the request went.
• HTTP headers to tell the content type15 of the returned data.
• A body that contains the response from the HTTP request.
An example record of this type could look something like this:
#jsfw_reply{
status=200,
headers="Content-Type: application/json",
body="This is a sample JSON string"
}
4.3
JSFW JavaScript API
The JSFW JavaScript API provides the components with an object oriented way
of constructing front-ends. At the top level a Body object is created to which
elements are added.
var body = new Body();
14
15
http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
13
This object is actually just an abstraction layer on top of a regular <div> element
selected by jQuery which acts as a container for the added elements. The elements
available are common HTML elements such as <table>, <ul>, <span> and <div>
and they are created and added to the Body element in a standardized manner.
Here a <table> with column headers TableHeader1 and TableHeader2 is created
and added to the Body.
var table = new Table(["TableHeader1, TableHeader2"]);
body.add(table);
The Body object also provides a function for filling those HTML elements with
JSON data from a given source in the form of the fill json(source, element)
function. Based on what type of element is filled, the JSON data is traversed
accordingly. This functionality is demonstrated with an example below, where the
<table> element table is filled with JSON from "/json/users" (which would be
the output of the get json() function of the users component’s Erlang module).
fill_json("/json/users", table);
4.4
JSFW Erlang API
Since the Erlang part of components only need to transfer data from the back-end
to the front-end the Erlang API is very limited and only consists of a single function, arg to pid/1, that is used to convert binary arguments to existing Erlang
process ids. In one of the example components, information about processes is displayed. A user can request information for a specific process based on its process
id by clicking on a link to that process in the list of processes. As an example, it
could be the process with process id 3. The Erlang backend would then be sent
<<"3">> as input to its get json/1 function.
arg_to_pid(<<"3">>).
The above example would then return the process id <<3>> which can be used by
the Erlang module to retrieve additional information about the specified process.
4.5
Documentation
A rather extensive documentation on how to use JSFW and what API functions
are available is included in the package itself. This was actually implemented as a
JSFW component and is accessible through the web interface.
14
4.6
JSFW Webserver Plugins
One of the goals with this framework is to have it easily integrate with di↵erent
web servers, and thus the need for a general web server layer arose. These plugins
are not actually too complex (see Appendix D) and they are supposed to translate
the incoming HTTP request to a uniform format that is supported and specified
by JSFW Core. When the framework has dealt with the incoming request, a reply
is returned to the plugin and it is up to the plugin to translate the response and
return it to the client. These plugins are written in Erlang.
4.7
JSFW Components
The components contain the actual functionality and this is where the user implements the specific functionalities. A component could for example list the online
users in a chat service. The components contain an Erlang module, a JavaScript
file and a CSS file. See Figure 2 for a simple illustration of the component data
flow.
Figure 2: Inner workings of JSFW components.
4.8
Erlang module
The role of the component’s Erlang module is to provide data that in turn will
be presented by the component’s JavaScript. Two standard functions have to be
implemented, namely get json/0 and get json/1. The former one should return
a tuple according to {ok, JSON} where JSON is a JSON encoding of the data that
the user wishes to provide to the front-end (e.g. a list of usernames and user id’s in
the case of a component that lists users). The second function has the same purpose
as the first one except that it also accepts an argument extracted from the trailing
part of the request URL (e.g. in the case http://hostname/json/users/3, the
Erlang binary <<"3">> will be sent as argument to the get json/1 function of the
users component).
15
4.9
JavaScript file
The JavaScript file contains a JavaScript object which consists of three standardized functions: init(), handle(argument) and refresh(). Below is an example
of a simple component.
var users = new function() {
var table;
this.init = function() {
var body = new Body();
table = new Table(["TableHeader1", "TableHeader2"]);
fill_json("/json/users", table);
body.add(table);
}
this.handle = function(argument) {
var body = new Body();
var span = new Span("Let’s display the argument: " + arg);
body.add(span);
}
this.refresh = function() {
fill_json("/json/users", table);
}
}
init() is called when the component is clicked in the menu and handles the
initial display of data on the front-end. Typically (and as illustrated by the example above), this function sets up the initial appearance of the component by
creating suitable elements and fetching the JSON output from the Erlang module.
handle(argument) is called when a link printed by a component is clicked. If the
link leads the user to http://hostname/users/3 then 3 is the argument that is
sent to the handle(argument) function. Generally speaking, handle(argument)
takes care of setting up the front-end when an argument is given. Automatic
refreshing of components is supported by JSFW Core and upon refreshing, it is
refresh() that is called.
4.10
CSS file
A CSS file was used to include any custom styling that the component may wish
to use.
16
4.11
Constructing a Component
This section will walk you through the process of constructing a simple component
that displays a list of usernames when we navigate to http://hostname/users.
Having constructed the users component also leads to exposing the web resources
http://hostname/json/users and http://hostname/json/users/Argument, which
gives us the data returned from users:get json() and users:get json(Argument)
respectively.
users.erl
The Erlang module’s role is to JSON encode and output the data that we want to
display, in this case the list of users: ["John", "Jackie", "Michael", "Sarah"].
We ignore the get json(Arg) function for this simple example.
- module ( users ).
- export ( [ get_json /0 , get_json /1 ] ).
get_json () ->
Data = [ " John " , " Jackie " , " Michael " , " Sarah " ] ,
jsfw_json : encode ( Data ).
get_json ( Arg ) -> jsfw_json : encode ( Arg ).
users.js
The role of the JavaScript file is to take the JSON data output by the Erlang module and use the JavaScript API to put that data in an HTML element. We ignore
the handle and refresh functions for this simple example. For this example, this
means:
• Creating the Body and List elements
• Fetching the data from the JSON resource http://hostname/json/users
• Calling the API function fill json(source, target) to fill the list with
the users data
• Adding the list element to the body
var users = new function () {
this . init = function () {
var body = new Body ();
var list = new List ();
fill_json ( " / json / users " , list );
body . add ( list );
17
}
this . handle = function ( argument ) { }
this . refresh = function () { }
}
Result
This will result in an <ul> element being populated by the four names output by
the Erlang module above.
<ul >
<li > John </ li >
<li > Jackie </ li >
<li > Michael </ li >
<li > Sarah </ li >
</ ul >
5
Evaluation
When evaluating a framework like this, one cannot expect to get any raw benchmark results but instead has to consider a di↵erent approach. Since the goal is
an easy to use framework one should try to evaluate its user-friendliness. How
does one do this in a reasonable manner? Although this framework is not a user
interface per se, but rather a tool to aid developers in creating user interfaces, we
still believe some of the principles from user interface evaluation are applicable.
Selected methods from a few papers on the topic [12, 13] are described below.
5.1
5.1.1
Evaluation Methods
Heuristic Evaluation
In this type of evaluation, a number of evaluators (three to five people [14]) are
presented with an interface design and are asked to comment on it, based on a list
of usability principles (heuristics) [14]. It is up to the evaluators how they want to
proceed with the evaluation, as long as it is conducted individually. The results
of the di↵erent evaluations are then aggregated to a single final evaluation. It is
suggested that this type of evaluation performs better when UI experts are used
as evaluators [14] and this is confirmed by results of research papers comparing
di↵erent evaluation methods [15, 13]. An advantage of heuristic evaluation is that
the evaluators are not performing real tasks per se and it can therefore be applied
at a very early stage of implementation.
18
5.1.2
Usability Testing
A di↵erent approach is to present evaluators (UI experts or not) with pre-defined
user scenarios and let them perform the tasks and report any errors encountered
or strange behavior by the system. Usability testing usually cannot be applied
until at a later stage in product development since one needs to have the features
in place required for the evaluators to perform their tasks.
5.1.3
Empirical Evaluation
In empirical evaluation, potential users test the interface. In contrast to usability
testing, the users are free to use the interface in the way that they normally would
and comment on it accordingly. This type of evaluation su↵ers from the same
problem as usability testing, it cannot be applied early on in the development
cycle.
5.2
Evaluating JSFW
When evaluating JSFW, as previously mentioned, one does not evaluate a standard user interface but instead looks at an interface made up of API functions
and an API documentation that are used to construct customized user interfaces.
Since UI experts are not available for this evaluation, letting potential users (i.e.
Erlang developers) conduct the evaluation seems to be the most feasible alternative. Heuristic evaluation focuses on design flaws in user interfaces and since
we are more interested in the functionality of the framework, this method is not
suitable. Since we want users of this framework to be able to implement specific
functionality that they need in order to monitor their system, providing the testers
with scenarios containing general tasks (such as creating a new component) seems
reasonable. The advantage of usability testing over empirical evaluation is that
users are guaranteed to test particular tasks that we find interesting, while also
allowing them to get a general impression of the framework.
Clarke [16] describes the evaluation of APIs as finding the di↵erences between
what users expect from the API and what the API really does. It is mentioned
performing this evaluation with the help of the cognitive dimensions framework [17]
which lists 12 dimensions/factors that are important when it comes to API design.
These dimensions are good pointers towards things to consider when constructing
APIs and therefore they will be useful in the evaluation as well.
5.2.1
Cognitive Dimensions
• Abstraction level. The minimum and maximum levels of abstraction exposed
by the API, and the minimum and maximum levels usable by a targeted
19
developer.
• Learning style. The learning requirements posed by the API, and the learning
styles available to a targeted developer.
• Working framework. The size of the conceptual chunk (developer working
set) needed to work e↵ectively.
• Work-step unit. How much of a programming task must/can be completed in
a single step.
• Progressive evaluation. To what extent partially completed code can be executed to obtain feedback on code behavior.
• Premature commitment. The amount of decisions that developers have to
make when writing code for a given scenario and the consequences of those
decisions.
• Penetrability. How the API facilitates exploration, analysis, and understanding of its components, and how targeted developers go about retrieving what
is needed.
• API elaboration. The extent to which the API must be adapted to meet the
needs of targeted developers.
• API viscosity. The barriers to change inherent in the API, and how much
e↵ort a targeted developer needs to expend to make a change.
• Consistency. How much of the rest of an API can be inferred once part of it
is learned.
• Role expressiveness. How apparent the relationship is between each component exposed by an API and the program as a whole.
• Domain correspondence. How clearly the API components map to the domain
and any special tricks that the developer needs to be aware of to accomplish
some functionality.
5.2.2
Method
Usability tests combined with the help of cognitive dimensions framework were
used to conduct user testing from people within the Erlang community. This is
the most relevant group of people since they are the potential users of JSFW. In
practice, JSFW was shipped to developers together with an API documentation,
20
a set of tasks that we wanted to be tested and a link to an online evaluation
form consisting of questions based on the cognitive dimensions framework. Details
about the user scenarios and the questions asked can be found in Appendix A and
Appendix B respectively.
5.3
Results
The evaluation package was sent out to seven developers, all working as Erlang
developers.
5.3.1
Identified Problems
The testers found the following critical problems.
1. Not enough feedback was given to the user when constructing components.
2. The API documentation did not contain enough examples.
3. It was not obvious to tell what data structure to return from the Erlang
modules of components.
4. A need for updating only parts of components was presented.
5. Reloading of components did not function every time.
6. The ability to send input to components was sought after.
5.4
Resolution of Problems
After having researched methods for resolving the problems identified by the developers, the framework was changed in the ways described below.
Not enough feedback was given to the user when constructing components
This problem belongs to the progressive evaluation dimension and makes it difficult
for users to to test partially completed code while implementing a new component.
In order to resolve this, the framework was changed to report component compilation and runtime errors to the component front end.
21
The API documentation did not contain enough examples
An insufficient documentation is a type of problem that belongs to the role expressiveness dimension since one purpose of the documentation is to tell the user how
each API part fits into the program as a whole. Including examples is a good way
to make a documentation better. Thus, examples were added to the framework
documentation.
It was not obvious to tell what data structure to return from the Erlang
modules of components
Once again, this is a problem related to an insufficient documentation and the
documentation was updated to account for this accordingly.
A need for updating only parts of components was presented
This problem does not fit accurately into any of the dimensions but is clearly a
problem concerning usage limitation. In order to be more flexible, the framework
was changed to use jQuery selectors16 as internal selectors when specifying targets
for data to be put in.
Reloading of components did not function every time
The old way of recompiling components (via the settings interface) was changed
into an ”on demand” method that recompiles the components when the user navigates to them. A timestamp check was implemented to make sure compilation
only occurs if the .erl file is newer than the .beam file. This was an improvement
over the old implementation which had a problem with purging old code causing
it to sometimes not pick up changes.
The ability to send input to components was sought after
In response to this a new template tag was introduced that prints out a text input
to allow users to send arbitrary Erlang terms as input to components.
Other changes
In addition to resolving problems identified by the testers, new features were implemented. The component structure was changed to also include two HTML
templates to allow for more flexible and less painful front-end presentation. This
16
http://api.jquery.com/category/selectors/
22
was implemented statically at this stage with ideas to specify any number of templates in a future version.
6
Conclusion and discussion
A fairly competent and complete framework prototype was implemented and was
well received by the evaluators. It also became obvious, with the help of the
web server abstraction layer, that supporting multiple web servers was straight
forward and did not require much code at all since di↵erent web servers have a
lot in common. The evaluation performed was very helpful to identify usability
problems and thanks to that, it became clear what changes were necessary to make
the framework more usable.
6.1
Challenges
While usability testing really helped the implementation of the framework it also
showed how long it can take to get a hold of people and actually get a response.
The original idea was to perform the evaluation in several iterations with implementation adjustments in between, but this proved to take too much time.
7
Future Work
7.1
WebSockets
WebSockets17 is an interesting new concept that allows for a two-way communication between client and server, resulting in the ability for the server to push data
to the front-end without the client needing to request it. With the help of this
we can for example update graphs only when new data is available on the server
without having the client poll the server at regular intervals. An implementation
of this would include adding a web socket client abstraction layer for components
to use.
7.2
Support for Additional Webservers
While developing your own JSFW Web server plugin is fairly simple it would
certainly make JSFW more accessible if support for additional web server was
included. We could even imagine the scenario where the JavaScript front-end is
17
http://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-17
23
separated from the back-end, making it possible to run JSFW on non-Erlang server
software (e.g. Apache18 ).
7.3
Integration of a template engine
Many powerful template engines exist that allow a lot of flexibility and customizability of dynamic data in HTML templates. One good example is ErlyDTL19
which is an Erlang implementation of the Django Template Language20 .
8
References
[1] Ericsson AB. WebTool. http://www.erlang.org/doc/man/webtool.html.
Accessed: November 2011.
[2] Ericsson AB. Appmon. http://www.erlang.org/doc/man/appmon.html.
Accessed: November 2011.
[3] Spawnfest: 48 hour Erlang development competition. http://spawnfest.
com/, July 2011. Accessed: November 2011.
[4] Smells like BEAM spirit. BigWig: A better Erlang webtool. http:
//www.metabrew.com/article/bigwig-erlang-webtool-spawnfest, July
2011. Accessed: November 2011.
[5] J. Armstrong. Concurrency Oriented Programming in Erlang. Challenge,
2002.
[6] J. Armstrong. Making Reliable Distributed Systems in the Presence of Software Errors. http://www.erlang.org/download/armstrong_thesis_2003.
pdf, 2003.
[7] A. Sullivan. JSX - JSON library for Erlang. https://github.com/
talentdeficit/jsx, October 2010. Accessed: November 2011.
[8] Smells like BEAM spirit. JSX - JSON library for Erlang. https://github.
com/spawnfest/jsx, July 2011. Accessed: November 2011.
18
http://httpd.apache.org/
https://github.com/evanmiller/erlydtl
20
https://docs.djangoproject.com/en/dev/ref/templates/builtins
19
24
[9] M. Troug. Erlang JSON libraries benchmarks. https://github.com/
okeuday/erlbench, November 2011. Accessed: November 2011.
[10] M. Troug. Erlang JSON libraries benchmark results. http://okeuday.
livejournal.com/18353.html, March 2011. Accessed: November 2011.
[11] B. Ippolito. Mochiweb, an Erlang library for building lightweight HTTP
servers. https://github.com/mochi/MochiWeb, November 2011. Accessed:
November 2011.
[12] Jakob Nielsen. Usability inspection methods. In Conference companion on
Human factors in computing systems, CHI ’95, pages 377–378, New York,
NY, USA, 1995. ACM.
[13] Robin Je↵ries, James R. Miller, Cathleen Wharton, and Kathy Uyeda. User
interface evaluation in the real world: a comparison of four techniques. In
Proceedings of the SIGCHI conference on Human factors in computing systems: Reaching through technology, CHI ’91, pages 119–124, New York, NY,
USA, 1991. ACM.
[14] Jakob Nielsen and Rolf Molich. Heuristic evaluation of user interfaces. In
Proceedings of the SIGCHI conference on Human factors in computing systems: Empowering people, CHI ’90, pages 249–256, New York, NY, USA,
1990. ACM.
[15] Jakob Nielsen. Finding usability problems through heuristic evaluation. In
Proceedings of the SIGCHI conference on Human factors in computing systems, CHI ’92, pages 373–380, New York, NY, USA, 1992. ACM.
[16] S. Clarke. Measuring API Usability. Dr. Dobbs Journal, May 2004.
[17] T. R. G. Green and M. Petre. Usability analysis of visual programming
environments: a ‘cognitive dimensions’ framework. JOURNAL OF VISUAL
LANGUAGES AND COMPUTING, 7:131–174, 1996.
25
Appendix
A
Scenarios
This is a short introduction followed by the scenarios sent out to the developers
that tested the framework.
A.1
Introduction
This framework aims to help with the development of web views that can help
when monitoring Erlang nodes. Users have the possibility to create custom made
components in the framework that display relevant information. A JavaScript
API exists that helps with the presentation of JSON data sent from Erlang modules. This evaluation presents a number of scenarios and questions that would
be valuable to us when improving/evaluating the framework. Documentation can
be found at http://localhost:8080/jsfw_docs once the web server is started.
Along with the examples found in the framework, this should be enough information to develop fairly advanced components.
In order to evaluate we would like you to try out the scenarios listed below
and after having completed them (or failed them for that matter), answering the
questions below.
A.2
A.2.1
Scenarios
List component
• Create a new component (mylist) through the settings interface
• Confirm that the component was created successfully by refreshing the page
and visiting the newly created component
• Modify the component so that it prints a list of names in an unordered list
(<ul>)
A.2.2
Table component
• Create a new component (mytable) through the settings interface
• Confirm that the component was created successfully by refreshing the page
and visiting the newly created component
• Modify the component so that it prints some tabular data to a table (<table>)
26
A.2.3
Refresh
• Modify the mytable component so that the displayed data gets refreshed
• Add a button to pause/resume the refreshing
A.2.4
Custom
• Create a new component (mycomponent) in the settings interface
• Modify this component into a component of your own choice
A.3
Questions
Questions are found online here: http://goo.gl/bUUn8
B
Questions
• What is your knowledge of Erlang? 1-5
• What is your knowledge of JavaScript/HTML? 1-5
• What browser are you using (and version)?
• What operating system are you running?
• What version of Erlang are you running?
• Could you complete the listed scenarios? If no, what hindered you?
• Did you feel that you could test your component even if it wasn’t finished?
If no, what more feedback would you want?
• Did you feel that you could implement your component in an iterative manner?
• When using the API, is it is easy to know what methods to use?
• Does the object oriented way of building the JavaScript part of components
make sense?
• Is the amount of code needed to construct a TABLE element suitable?
• Once you managed to create the LIST, was it easy to construct the TABLE?
• Was there something in the framework that made you feel limited?
27
• Did you miss some API methods?
• Was it difficult to know what to output from the Erlang module?
• What component did you try to implement for the Custom scenario?
• Were you successful in this? Did you think of another idea and considered
it too difficult to implement using this framework?
C
C.1
Framework Documenation
Structure
In the root of the program where you want to integrate JSFW, there must exist a
file named jsfw.config, which must contain tuples according to:
{webserver_plugins_dir, PATH}
PATH - string, the path to the directory holding the webserver plugins
{components_dir, PATH}
PATH - string, the path to the directory holding the components
C.2
URL Scheme
The framework relies on a few static URLs and a URL system that are described
below.
/static/RESOURCEPATH
Returns pre-shipped resource file found in priv/RESOURCEPATH
/json/components
Returns a JSON array containing available components and their attributes from
their respective component.config files
/json/COMPONENT
Returns JSON encoding of COMPONENT:get()
/json/COMPONENT/ARG
Returns JSON encoding of COMPONENT:get(ARG)
28
C.3
Component structure
Components are placed in the components dir directory as specified in the config
file mentioned above, in a directory named component. This here (jsfw docs)
is actually a component just like any other! Individual components consist of a
JavaScript file, an Erlang module, a HTML template, another HTML template
to display arguments, a CSS file and are structured and named in the following
manner:
C.3.1
HTML template (COMPONENT.html)
1| <h2>%%COMPONENT%%</h2>
2| Click this example argument link: <span id="jsoncontainer"></span>
This is the template used when creating a new component. All occurrences of
%%COMPONENT%% are then replaced by the name of the new component.
Support exists for a few special template tags and those are described below.
• {{toggle button}} - will be converted to a toggle button to pause/resume
the automatic refreshing of a component (refreshing only happens if the
javascript file implements the refresh function).
• {{input}} - will be converted to a text input that allows you to send arbitrary Erlang terms to your component. These terms will eventually be
handled by the component’s Erlang module, more specifically, the input/1
function. This text input should allow most Erlang terms.
C.3.2
Handle template (handletemplate.html)
1| <h1>Handle template for component: %%COMPONENT%%</h1>
2| Argument given: <span id="argcontainer"></span>
This is the template shown when displaying a component with an argument (i.e.
path /COMPONENT/ARG)
C.3.3
JavaScript file (COMPONENT.js)
var %%COMPONENT%% = new function() {
// This function is called when the component link is
// clicked in the menu
this.init = function() {
fill_json("/json/%%COMPONENT%%", $("#jsoncontainer"));
29
}
// This function is called when the component is
// automatically refreshed
this.refresh = function() {}
// This function is called when an argument is
// clicked on the component’s page
this.handle = function(arg) {
fill_json("/json/%%COMPONENT%%/" + arg, $("#argcontainer"));
}
}
This is the template used when creating a new component. All occurrences of
%%COMPONENT%% are then replaced by the name of the new component. The
JavaScript file needs to uphold this basic structure, with the variable declaration on the top level as well as the init function declaration and the (optional)
refresh function declaration for automatic refreshing of the component. See how
jQuery selectors are used to define where to put the JSON data?
C.3.4
Erlang module (COMPONENT.erl)
-module(%%COMPONENT%%).
-export([get/0, get/1]).
% -export([init/0]).
% -export([input/1]).
get() ->
[{’$link’,<<"This is an example argument">>,<<"/%%COMPONENT%%/3">>}].
get(Arg) ->
Arg.
% init() ->
%
ok.
%
% input(Input) ->
%
io:format("Component ~p received input: ~p~n",
[%%COMPONENT%%, Input]).
This is the template used when creating a new component. All occurrences of
%%COMPONENT%% are then replaced by the name of the new component. Note how
30
get/0 and get/1 should return data that can be encoded to JSON (see more
information further down). init/0 and input/1 are optional. init/0 is called
when the module is loaded the first time or recompiled and is used for setup
(such as registering a spawned process etc.). input/1 is called whenever input
is sent to the module with the help of the {{input}} template tag (see HTML
documentation).
C.3.5
Config file (component.config)
[{enabled,"true"},{displayname,DISPLAYNAME}]
This file contains the properties specified for the component in question. The value
of displayname is a string that is displayed in the component menu and enabled
decides whether it should show in the menu or not.
C.3.6
CSS file (COMPONENT.css)
This is optional and contains any custom styling wanted for the component.
Components can be created and enabled/disabled through the /jsfw settings web
interface. Go ahead and create your first example component right away!
C.4
JavaScript API
There is actually only a single JavaScript API function but that should help you
complete most of your implementations if you’ve constructed a clever HTML template.
fill_json(source, element);
Example: fill_json("/json/users", $("#usertable"));
Used to fill element (which is an element selected using jQuery selectors (more
on those here: http://api.jquery.com/category/selectors/) with the JSON
returned from the URL specified by source (typically ”/json/COMPONENT”).
Make sure that you format your output Erlang data so that it suits the element
that you wish to put it in (see more on that under Erlang API). In the above
example, the data returned from /json/users is inserted into the html ¡table¿
element with id ’usertable’. Thus, the Erlang data output from users:get()
must be in tabular format.
31
C.5
C.5.1
Erlang API
jsfw utility
arg_to_pid(Arg)
Given a binary Arg (e.g. <<"0.1.0">>), returns a PID (<<0.1.0>>)
C.5.2
Data structures
Below is a list of how Erlang data should be structured. This is what should
be returned from your component’s get() and get(Arg) functions. When using
fill json(), JSFW will look at the target element type to decide how to iterate
over the data given to insert it.
C.5.3
Tabular data (output in a <table>)
[[row1col1, row1col2, ...], [row2col1, row2col2, ...], ...]
C.5.4
List data (output in a <ul>)
[element1, element2, element3, ...]
C.5.5
Links
{’$link’, TEXT, URL}.
Example: {’$link’,<<"3">>,<<"/users/3">>}
This will be printed as:
<a href="/users/3">3</a>
Clicking on that link when navigating the users component will call the optional handle(argument) javascript function of the component with ”3” as argument (in this particular case). Fetching the JSON from /json/users/3 will call
users:get(<<”3”>>) similarily.
C.5.6
PIDs
PIDs are treated with special care and taken care of by the included jsfw pid
component. All PIDs printed will be links to /jsfw pid/PID and when clicked, be
displayed by the jsfw pid component.
32
D
D.1
Source Code
Cowboy Web server plugin
- module ( jsfw_cowboy ).
- include_lib ( " jsfw / include / jsfw . hrl " ).
- include_lib ( " cowboy / include / http . hrl " ).
- behaviour ( cowboy_http_hand l er ).
- export ( [ start /0 ] ).
- export ( [ init /3 , handle /2 , terminate /2 ] ).
init ( { _Any , http } , Req , [ ] ) ->
{ ok , Req , undefined } .
handle ( Req , State ) ->
JSFWreq = to_jsfw_request ( Req ) ,
{ ok , JSFWreply } = jsfw_request : handle_request ( JSFWreq ) ,
StatusCode = jsfw_reply : get ( status , JSFWreply ) ,
Headers = jsfw_reply : get ( headers , JSFWreply ) ,
Body = jsfw_reply : get ( body , JSFWreply ) ,
{ ok , Req2 } = cowboy_http_req : reply ( StatusCode , Headers , Body , Req ) ,
{ ok , Req2 , State } .
% % " Translates " a cowboy http request
% % to a jsfw_req as defined in jsfw . hrl
- spec to_jsfw_request (# http_req { } ) -> # jsfw_req { } .
to_jsfw_request ( Req ) ->
% % Extract the relevant information from the Cowboy HTTP request
{ Method , Req } = cowboy_http_req : method ( Req ) ,
{ Raw_Path , Req } = cowboy_http_req : raw_path ( Req ) ,
{ Args , _Req } = cowboy_http_req : qs_vals ( Req ) ,
% % Constructs the jsfw_req record
# jsfw_req {
% % HTTP Method used
method = Method ,
% % Turn the binary path into a list of binaries ,
% % strip empty binaries
path = [ B || B <- re : split ( Raw_Path , " / " ) , B /= <<>> ] ,
% % Query string arguments
args = [ { list_to_atom ( binary_to_list ( Key )) , Value } ||
{ Key , Value } <- Args ]
}.
terminate ( _Req , _State ) ->
ok .
start () ->
ok .
D.2
Misultin Web server plugin
- module ( jsfw_misultin ).
33
- include_lib ( " jsfw / include / jsfw . hrl " ).
- include_lib ( " misultin / include / misultin . hrl " ).
- export ( [ handle_http /1 ] ).
handle_http ( Req ) ->
% % Translate the misultin request
JSFWreq = to_jsfw_request ( Req ) ,
{ ok , JSFWreply } = jsfw_request : handle_request ( JSFWreq ) ,
Status = jsfw_reply : get ( status , JSFWreply ) ,
Headers = jsfw_reply : get ( headers , JSFWreply ) ,
Body = jsfw_reply : get ( body , JSFWreply ) ,
Req : respond ( Status , Headers , Body ).
% % Extracts the relevant information from the misultin http request
% % and returns a corresponding record structure as defined by jsfw . hrl
- spec to_jsfw_request (# req { } ) -> # jsfw_req { } .
to_jsfw_request ( Req ) ->
Method = Req : get ( method ) ,
{ abs_path , Raw_Path } = Req : get ( uri ) ,
Args = Req : parse_qs () ,
# jsfw_req {
method = Method ,
% % Turn the string list into a list of binaries
path = [ list_to_binary ( P ) || P <- string : tokens ( Raw_Path , " / " ) ] ,
args = [ { list_to_atom ( Key ) , list_to_binary ( Value ) } ||
{ Key , Value } <- Args ]
}.
D.3
D.3.1
Example Components
List Processes
- module ( lister ).
- export ( [ get /0 , get /1 ] ).
get () ->
PIDs = erlang : processes () ,
Key = fun (K , PID ) -> proplists : get_value (K , erlang : process_info ( PID ))
end ,
[[
% % Return Process ID , registered name , current function
PID ,
proplists : get_value ( registered_name , PID ) ,
proplists : get_value ( current_function , PID )
] || PID <- PIDs ] .
get ( Arg ) ->
PID = jsfw_utility : arg_to_pid ( Arg ) ,
Info = erlang : process_info ( PID ) ,
[ [ Key , Value ] || { Key , Value } <- Info ] .
34