Cloud Computing and
Architecture
SkyCave
Domain
• SkyCave is a
– Massive multi-user online, exploration and creation
experience with a bit of social networking
– … with a horrible user interface
– … and high disregards for security
• Learn that in other courses…
• Inspired by the very first ‘interactive fiction’ game
for a computer: Colossal Cave Adventure
– Will Crowder, 1972
• I played my first game in 1986
CS@AU
Henrik Bærbak Christensen
2
The History
CS@AU
Henrik Bærbak Christensen
3
SkyCave is…
• … changed as it
– Removes all adventure aspects
• ‘get lamp’ ‘throw axe’ ‘use key’
– Allows modifications to cave which is a (x,y,z) lattice
• ‘dig n You are in a maze with twisty passages’
– Will create a new room, north of this, with the given description
– (very limited) social networking through the ‘wall’
• Post and read messages on each room’s single wall
– Massive online / distributed system (MMO)
• Not one but 10, 1000(!), 1.000.000 (!!!) players
– Single sign-on
• Once registered, each player can access every cave
CS@AU
Henrik Bærbak Christensen
4
SkyCave is…
• … changed as it
– Has some weird features with a learning focus
• ‘weather’
report real outside weather
– Is highly (re)configurable
• To support automated testing using test doubles
• … and therefore support test-driven development
• To support incremental architectural work
– HashMap → MongoDB
– Local call → Socket → RabbitMQ
• To support architectural testing
– Saboteur implementation of central services
CS@AU
Henrik Bærbak Christensen
5
SkyCave is…
• … in Release 2.0 
– The architecture is overall sound, believe it will stand
the test of time…
– The code is well tested
– But…
– There may still be bugs that I do not know of
– There are known performance and scalability issues
– There are known hacks, omissions, pending refacts.
CS@AU
Henrik Bærbak Christensen
6
Deployment Viewpoint
• The handed out SkyCave (‘socket.cpf’)
CS@AU
Henrik Bærbak Christensen
7
Deployment Viewpoint
• A 12+ grade solution for the SkyCave exam
CS@AU
Henrik Bærbak Christensen
8
DevOps on SkyCave
Development & Operations
Environments
• Three Environments are required
– Development environment
• Development tools – coding and testing
• Your labtop
– Staging environment
• Simulated production environment
• Virtual environment on you laptop or some VM Host
– Production Environment
• Real production environment
• EC2, Google, Digital Ocean, Azure…
CS@AU
Henrik Bærbak Christensen
10
Dev Env
• Either
– Local Install
• Get all the tools running on your laptop
– Andromeda
• Run Lubuntu Development machine using VMWare Player
• Find all information on the Tools web page
CS@AU
Henrik Bærbak Christensen
11
Staging Evn
• We will use Docker for staging and production
• Docker is a learning goal on its own right, on the
exercises for the first couple of weeks
CS@AU
Henrik Bærbak Christensen
12
Production Env
• Your SkyCave system must be running in the
cloud
• Pick your own supplier
– Google Amazon Microsoft offer free small VMs (last
time I checked)
– I recommend another, hazzlefree, supplier but it costs
around 10$ per month…
CS@AU
Henrik Bærbak Christensen
13
SkyCave Code Base
• Find the code base at the Tools page
• Solve the Iteration 0 exercises which as mostly
learning exercises…
• Read the SkyCave doc web page
• Read the Broker pattern chapter in FRS
• ‘ant javadoc’ and read the JavaDoc
• ‘ant coverage’ and browse the source code
CS@AU
Henrik Bærbak Christensen
14
Various Tips and Tricks
Slides from the E15 instance
(and slightly wrong)
CS@AU
Henrik Bærbak Christensen
15
SkyCave Dev Env
• Use any IDE you want for Dev
– I use Eclipse for no other reason than history…
• Ops is not using an IDE!
– You bash around in the shell (sorry for the pun)
• Morale: Ant is the real execution tool
– And even in DevOps I escape from IDE to the shell all
the time to get things done…
• Probably can be made from Eclipse, but … history…
CS@AU
Henrik Bærbak Christensen
16
Again: Ant is right, Eclipse is not!
CS@AU
Henrik Bærbak Christensen
First Light
• [Demo in ‘CloudDev’]
– Download zip
– Unzip into ~/cave
– Resolve the dependencies / Ivy at work
• ant resolve [Explain that IDE will NOT work without it]
– Run the tests [Explain ‘TEST-RESULT’]
• Hey – we are intensively testing a distributed system on a
single machine configuration?
• Rather cool, isn’t it?
– Review test quality / JaCoCo at work
• ant coverage [Explain color coding]
CS@AU
Henrik Bærbak Christensen
18
First Light
• Getting an IDE up and running
– The ‘eclipse’ import trick
• Namely – never use the import
feature 
• Do
– New Java Project
– Unclick ‘default location’
– Browse to the project root folder
– Done.
CS@AU
Henrik Bærbak Christensen
19
First Light
• Build path is always tricky
• Procedure
– Refresh project
– Right click project
– Menu ‘Build path’
‘Configure Build path’
– Pane ‘libraries’
• Add Jars
– Browse to cave/lib
• (default ivy folder)
• Add all
• Project/clean
CS@AU
Henrik Bærbak Christensen
20
Ivy
• You need ‘MongoDB’?
– Ivy pulls from
mvnrepository.com
Use ‘search’ on mvnrepository.com.
Click on wanted version.
Click on the ‘ivy’ pane.
Cut the <dependency> tag into
‘ivy.xml’
Run ‘ant resolve’
Good to go (for ant)
For Eclipse, redo the build
path configure process after
ivy has fetched it!
CS@AU
Henrik Bærbak Christensen
21
First Distributed Light
• After all – SkyCave it a multi-user system
• A simple implementation of the IPC interfaces
are provided
– Socket based – single application server
– Awfully slow on (my?) Windows!
• First we have to configure SkyCave
– setupLocalSocket.(bat/sh)
• Both in a client and server window
– Sets environment variables that define how abstract
factories for client and server inject dependencies
CS@AU
Henrik Bærbak Christensen
22
Disclaimer…
• Ideally, SkyCave should have a module
viewpoint divided into three modules
– Client module – for client side deployment
– Server module – for server side deployment
– Common module – for both
• Ant+Ivy module support add tons of complexity
to build scripts 
• Solved by avoidance!
CS@AU
Henrik Bærbak Christensen
23
Demo
CS@AU
Henrik Bærbak Christensen
24
Software Architecture
Domain model
• A very simple domain model !
– (And a good example of the fact that domain modeling
helps very little in designing a strong architecture!)
CS@AU
Henrik Bærbak Christensen
26
Viewpoints
• Software Architecture in 3 secs
– An architecture is not a single thing to be expressed in
a single diagram
– Minimal viewpoints
• Module viewpoint:
static information
– Classes, interfaces, packages, inherits, depends upon, …
• Component-Connector viewpoint: dynamic information
– Objects, threads, executes, calls, …
• Deployment viewpoint:
topology information
– Computers, processes, networks
– [Bass et al., Christensen 2012]
CS@AU
Henrik Bærbak Christensen
27
Deployment Viewpoint
The one IBM starts with!
Why? Cost of production!
Deployment Viewpoint
• The central nodes in SkyCave / Three Tier + MS
CS@AU
Henrik Bærbak Christensen
29
Deployment Viewpoint
• The handed out SkyCave (in one config)
CS@AU
Henrik Bærbak Christensen
30
Deployment Viewpoint
• A 12+ grade solution for the SkyCave exam
CS@AU
Henrik Bærbak Christensen
31
Deployment Viewpoint
• An expected 02 grade minimal solution
CS@AU
Henrik Bærbak Christensen
32
The Hardware side of it?
• Do not dispair.
CS@AU
Henrik Bærbak Christensen
33
Component Connector Viewpoint
Executing software earns money.
UML diagrams do not…
CC / Dynamic View
• Dynamics, yeah…
• Scenario
– Mikkel moves NORTH and then LOOKs around.
• Exercise
– Sketch a sequence diagram!
CS@AU
Henrik Bærbak Christensen
35
A first shot…
• Comments? Issues?
– Multi user… Persistence.
CS@AU
Henrik Bærbak Christensen
36
Persistence
• OO databases never really made it…
– Usually a strong indicator that it is a bad idea!
CS@AU
Henrik Bærbak Christensen
37
PODO
• The Object / DB mismatch
– Objects have state and behavior
– Data objects have just state
• Record type / Plain Old Data Object
– Pascal Record / C struct
– Just data and accessor methods
– Suitable for networking and persistence
CS@AU
Henrik Bærbak Christensen
38
Networking…
• Aah – the Cmd object is on the client side!
Client
CS@AU
Back tier
Henrik Bærbak Christensen
39
Objects or Procedures?
• At the basic level, networking is just
– send(address, byte[])
– byte[] receive(address)
• Nasty! Abstraction to the rescue…
• RPC: Remote procedure call
– Call ‘String foo(int, String, double)’ on remote server
• RMI: Remote method call
– Call ‘myObj.foo(int, anotherObj*)’ on remote server
CS@AU
Henrik Bærbak Christensen
40
RMI had its glory…
• …period before the ‘big web thingy’ hit
– CORBA, Java RMI, DCOM, .Net remoting
• Heavy weight because
– Object references and ‘pass by reference’
– Registries for (object, server address+ID) lookup
• But HTTP and web servers, SOAP, REST are
– Basically remote procedure calls
– Simple data types, pass by value, no refs
CS@AU
Henrik Bærbak Christensen
41
RMI: Broker Pattern
• Indeed not trivial…
– Remote Method Invocation is an ‘easy’ way to get it
• Someone remembers the Java RMI solution?
– What are the constituent parts?
– Tool support?
CS@AU
Henrik Bærbak Christensen
42
Broker Pattern
• RMI hardwires protocol and communication scheme 
– SkyCave needs to be
• more flexible (support networking using sockets, MQ, HTTP?, …)
• more control of failure modes (time outs, networking issues, fail-over)
– … but does not need
• Object references (except ‘this’ = Mathilde player, not Mikkel)
• and pass by reference (params are simple types)
• Solution:
– Basically a RPC + a ‘this’ reference
– Code (parts of) Broker by hand
• Handle ‘player.move(north)’ type methods, but not more…
• Fine grained control, light weight, code size is not that scary
• …and demo that 
CS@AU
Henrik Bærbak Christensen
43
Broker [POSA v4]
• The basic dynamical architecture…
• Broker is actually a set of patterns; and not really
visible in the code itself
• SkyCave avoids the discovery and registration;
use a more direct lookup (using IPs)
CS@AU
Henrik Bærbak Christensen
44
Client Proxy
• Client Proxy
– Translate OO method calls to request objects
– Call the Client Request Handler
CS@AU
Henrik Bærbak Christensen
45
Client Request Handler
• Encapsulate IPC to the server
– (and we can use a ‘fake object’ instead!)
CS@AU
Henrik Bærbak Christensen
46
Invoker
• On server side…
– In SkyCave it is a slight variant
• Actually receives a JSON request object
• Does ‘dispatching’ from the Reactor pattern
– Lookup a ‘dispatcher’ for handling either Cave methods or
Player methods
CS@AU
Henrik Bærbak Christensen
47
Reactor
• Connects to the OS / IPC receiver end
– Blocking wait on incoming requests
– Dispatches to event handlers
• This is actually coded in SkyCave’s Invoker
CS@AU
Henrik Bærbak Christensen
48
Dynamics
• Overall the dynamics is
– Marshaler: Converting to/from JSON
CS@AU
Henrik Bærbak Christensen
49
Module viewpoint
Supports overview and testing.
Or not!
CS@AU
Henrik Bærbak Christensen
50
Three Tier Architecture
UI (client)
App Server (server)
Database / Ext services
(service)
Why is ‘doubles’ not in
test tree?
CS@AU
Common folders contain PODO’s shared
between any two layer.
Domain has core domain interfaces.
Config folders contains dependency injection
handling abstractions.
Doubles folders contain test doubles.
IPC contains marshalling + IPC exceptions.
Henrik Bærbak Christensen
51
Testability
Test-driven development.
Architectural Testing.
Compositional Design Principles
• Central technique to achieve reliability: Testing
• Automated testing require control of input and
output
• CDP is a prime technique
CS@AU
Henrik Bærbak Christensen
53
At all important points…
• … we need to inject replacement
implementations (Mezsaros: Test doubles)
– Cave with
• MongoDB storage or in memory storage (FakeObject)
• HTTP Weather service or stub (Test stub)
• IPC over socket, or … local method calling (FakeObject)
• Nygard: Integration points
– Integration Points: Points in the software that integrate
with external systems
CS@AU
Henrik Bærbak Christensen
54
Disclaimer
The injection architecture of SkyCave is a bit
over elaborated for historical reasons.
The ambition of this course is to go MO(O)C
and use automated architectural testing for
grading. This requires a very high flexibility for
the auto-grader (‘crunch’) to tweak your code
appropriately.
Sorry for the extra complexity .
CS@AU
Henrik Bærbak Christensen
55
Examples: Cave Configuration
TestDoubleFactory
CS@AU
Henrik Bærbak Christensen
56
Example: IPC Configuration
• In TestPlayerProxy we simulate IPC by local method calls
– Any method-call does full ‘broker-round-trip’
• Proxy call -> marshalling into JSON -> demarshalling -> dispatching
-> reply marshalling -> demarshalling -> proxy upcall
CS@AU
Henrik Bærbak Christensen
57
Wiring mechanism
• Dependency injection means dependencies are
injected
– Someone has to have this responsibility
• Abstract Factory
– Question: How to implement the concrete factories
– Potentially
• Java impl for every combination
• Read in property files
• Read environment variables
CS@AU
Henrik Bærbak Christensen
(comb. explosion)
(comb. explosion)
(same same )
58
Environment Variables
• Advocated by 12factor.net “Twelve factors § III”
CS@AU
Henrik Bærbak Christensen
59
Exercise
• Why using an instance of EnvReaderStrategy?
CS@AU
Henrik Bærbak Christensen
60
Back to the Env Variables…
• (Quite) easy to reconfigure
– E.g. call ‘setupLocalSocket.sh’ for socket based IPC
on localhost:37123
– Execute
• export SKYCAVE_APPSERVER=caveweather.baerbak.com:37123
– Now, next ‘ant cmd’ will contact my cloud based
instance of SkyCave daemon
• Give and
take a
crashed
server 
CS@AU
Henrik Bærbak Christensen
61
The Tool Stack
CS@AU
Henrik Bærbak Christensen
62
Required
• Java 7+, Ant, Ivy
– And an IDE
• Version control
– Subversion / Git / ?
• Docker 1.6+
• Bash / Ubuntu Linux
CS@AU
Henrik Bærbak Christensen
63
My Own Tool Stack
• Development Environment
– Windows 7+8
•
•
•
•
Eclipse Luna-Kepler
Emacs
SourceTree (git), TurtoiseSVN
Putty – pageant – puttygen (windows secure shell)
– And authorized_keys in .ssh
• VMWare Workstation / Player
• Docker only in Ubuntu Linux VMs
–
–
–
–
CS@AU
Private repo’s on BitBucket
DigitalOcean droplets
Slack
And a private cloud…
Henrik Bærbak Christensen
64