Some
Design
Patterns
Or
Best
Practices
1
Modern Internet Service Oriented Application Development –
Lecture 2: REST Details and Patterns
© Donald F. Ferguson, 2015. All rights reserved.
Resource
Basics
2
Modern Internet Service Oriented Application Development –
Lecture 2: REST Details and Patterns
© Donald F. Ferguson, 2015. All rights reserved.
Some URLs
Obvious URLs
•
Collections/Extents
–
–
–
–
•
…/staff
…/stores
…/payments
…/rentals
Returns all staff
Return all stores
Return all payments
Return all rentals
Instances
–
–
–
–
3
…/staff/21
…/stores/11
…/payments/9
…/rentals/6
Return employee 21
Return store 11
You get the picture
You get the picture
Modern Internet Service Oriented Application Development –
Lecture 2: REST Details and Patterns
© Donald F. Ferguson, 2015. All rights reserved.
Resource URL and Representation
• GET …/stores/21 maps to a row in a table with the following columns
–
–
–
–
Store_id
Manager_staff_id
Address_id
Last_update
• A straightforward JSON representation is
{store_id : 21, manager_staff_id : 11, address_id : 9,
last_update : 1960-01-01 23:03:20}
• There are several issues with this format. Three significant ones are:
– How to find the “manager” with id=11 is unclear. I need to know side information not present in the
response.
– SQL columns names are often “unclear.” I am not exactly sure what “manager_staff” means.
– Last_update is a SQL datatype. Understanding how to convert to a corresponding data type in my
client application requires knowing the underlying implementation is SQL.
4
Modern Internet Service Oriented Application Development –
Lecture 2: REST Details and Patterns
© Donald F. Ferguson, 2015. All rights reserved.
A Better Representation
• I would return something like
{store: {“data” : {id : 21}, “link” : {“rel” : “self”, “href” : “…/stores/21”}},
{manager :
{“data” : {“id” : 11, “name” : {“lastName” : “Ferguson”, “firstName : “Donald”},
{“link” : {“rel” : “storeManager”, “href” : “…/staff/11}}}
{address : {“link” : {“rel” : “storeAddress”, “href” : “…/address/9}},
{“timestamp” : “30-Jan-2015”}}
• Observations
– If I got the {…{}…} right in that, I will eat a JSON manual.
– I converted the timestamp into something more human/string friendly.
– Returning the value of a foreign key columns is wrong.
I returned the value and an explicit link to navigate.
– I can optionally return some of the data in reference for convenience (columns from the JOIN).
– I make the column names a little friendlier.
5
Modern Internet Service Oriented Application Development –
Lecture 2: REST Details and Patterns
© Donald F. Ferguson, 2015. All rights reserved.
Some Additional Observations
• …/staff/21 maps to a row in a table of the form
–
–
–
–
–
–
Staff_id
First_name
Last_name
Address_id
Picture
……
• Some suggestions
– Return a name object, not individual fields, i.e.
{ … …, “name” : {“last” : “Ferguson”, “first” : “Donald”}, … …}
– Image is a “BLOB” = “Binary Large Object.” I would return
{… …, “Picture” : {“rel”: “picture”, “href” : “…/staff/21/picture”}, … …}
6
Modern Internet Service Oriented Application Development –
Lecture 2: REST Details and Patterns
© Donald F. Ferguson, 2015. All rights reserved.
The BLOB!
Or just use Amazon S3
• A GET on
– …/staff/21/picture should return
– The same kind of thing that you get when you click on
– http://upload.wikimedia.org/wikipedia/commons/e/ee/Grumpy_Cat_by_Gage_Skidmore.jpg
Content-Type: image/jpeg
In the Express framework for node.js
(http://expressjs.com/api.html) you
can set this with the call
res.type(‘image/jpeg’)
7
Modern Internet Service Oriented Application Development –
Lecture 2: REST Details and Patterns
© Donald F. Ferguson, 2015. All rights reserved.
Let’s Take a Minute
Remember when I talked about REST  “Uniform Interface?” This actually means
• Resource based
– URIs identify (locate) individual resources.
– The resources are representations and not the underlying DB schema, file data, … …
• Manipulation through representations
– If I have a representation (may include headers)
– I have enough information to reread, modify, delete (subject to security permission)
the resource
– And do not need “side information,” e.g. knowing that {“manager_id” : 21} needs
to become …/staff/21.
• Self-descriptive: Each “message” contains enough information to process it,
e.g. MIME type
8
Modern Internet Service Oriented Application Development –
Lecture 2: REST Details and Patterns
© Donald F. Ferguson, 2015. All rights reserved.
Let’s Take a Minute
•
•
The “pointer” to a resource is the URI
The common operations in any data model are (CRUD):
–
–
–
–
•
Create
Retrieve
Update
Delete
Datamodels commonly have containment and reference relationships
–
Containment: Represent logically contained resources as resources, e.g.
–
–
–
–
–
…/store/21/manager returns the resource representation of the correct manager from the “Staff” table
…/staff/32/name/last returns “Ferguson”
Datamodels have types, which you represent using MIME types
–
–
–
9
Do not return {… …, “lastname” : “Ferguson”, “firstname” : “Donald”, …}
Just because the database represents it that way
When you really mean {“name” : {“last” : “Ferguson”, “first” : “Donald”}}
Reference: Return a typed link to the resource
Relationships are navigable, e.g.
–
–
•
= HTTP POST
= HTTP GET
= HTTP PUT
= HTTP DELETE
Application/json
Application/XML
Image/jpeg
 JSON
 XML
 JPG image
Modern Internet Service Oriented Application Development –
Lecture 2: REST Details and Patterns
© Donald F. Ferguson, 2015. All rights reserved.
This may be Obvious but …
How do you implement a service? Use a framework.
• For example, Node.js + Express
– Create the URL base at localhost:1234
– Specify the “routes”
–
–
–
–
app.get(‘URL sub-path’, module.function)
app.put(‘URL sub-path’, module.function)
app.post(‘URL sub-path’, module.function)
app.delete(‘URL sub-path’, module.function)
– The JavaScript modules and functions
– Perform CRUD on the underlying implementation of the representation, e.g. RDB, MongoDB
– And any associated logical like validating data, convert foreign keys to links, etc.
• There are many others, e.g. Sails, JAX-RS, …
10
Modern Internet Service Oriented Application Development –
Lecture 2: REST Details and Patterns
© Donald F. Ferguson, 2015. All rights reserved.
This may be Obvious but …
How do you call a service? Use a library.
• For example, jQuery in a browser
• And there are many others, e.g. JAX-RS Client API
11
Modern Internet Service Oriented Application Development –
Lecture 2: REST Details and Patterns
© Donald F. Ferguson, 2015. All rights reserved.
Collections
12
Modern Internet Service Oriented Application Development –
Lecture 2: REST Details and Patterns
© Donald F. Ferguson, 2015. All rights reserved.
Collections
• All datamodels have some form of collections, e.g.
– Relational database tables
– File system directories
– Java.util.Collections.List
• These map naturally into URIs
– …/staff
– …/stores
– …/addresses
• But there are a couple of challenges
– Locating things in a collection
– Paging or iterating through a collection
13
Modern Internet Service Oriented Application Development –
Lecture 2: REST Details and Patterns
© Donald F. Ferguson, 2015. All rights reserved.
Locating Things in a Collection
• The collection typically has a unique, primary key
– With a natural representation in URL/REST,
– .../staff/21
– …/stored/36
– Which returns a resource of the form data and links
• Some collections have non-unique, secondary keys with
– A less natural representation in URL/REST, e.g.
– .../staff?name.last=“Ferguson”&name.first=“Donald” that
– Maps to “SELECT * FROM staff WHERE last_name=“Ferguson” AND first_name=“Donald”;
– That is itself also a collection.
• Some collections support ad hoc, client specified queries
– With quasi-natural representation in URL/REST, e.g.
– …/addresses?q=“zipcode=12345||state=“California”
– Maps to “SELECT * FROM addresses WHERE zipcode=‘12345’ OR state=‘California’
– That is itself also a collection
14
Modern Internet Service Oriented Application Development –
Lecture 2: REST Details and Patterns
© Donald F. Ferguson, 2015. All rights reserved.
Some Observations
•
Why surface secondary keys and not just ad hoc query?
–
–
Some databases do not support arbitrary, ad hoc query
DBAs may
–
–
Preclude ad hoc queries for various reasons
And only surface predefined, parameterized, compiled template queries
–
•
You should
–
Define your query language in your model
–
–
–
–
–
And not just surface the underlying query language, e.g. SQL, MongoDB
Because the underlying language is the implementation, not a representation
And also invites things like SQL injection attacks
Parse, validate and map the query into
–
–
Actual implementation language
Actual underlying datamodel, e.g. RDB, MongoDB
And I use q=“….” as the representation of the query string to draw a distinction between
–
–
The ability to process ad hoc queries on a collection
Versus a predefined, fixed set of combinations
–
–
–
15
SELECT * FROM CUSTOMERS WHERE zipcode = <p1>  …/customers?zipcode=12345
…/customers?zipcode=12345 is correct
…/customers?zipcode<=12345 is not correct
…/customers?zipcode=12345&type=Gold is not correct
Modern Internet Service Oriented Application Development –
Lecture 2: REST Details and Patterns
© Donald F. Ferguson, 2015. All rights reserved.
Iteration/Pagination – The Concept
• …/earth/animals/humans will return about 6 billion resources
• And I doubt very much that
– The client really wants all 6 billion in one call
– Or that the server really wants to return all six billion
• Which is where iterators and pagination occur
– Relational database
– DECLARE human_cursor FOR SELECT * FROM humans;
– OPEN human_cursor;
– FETCH human_cursor INTO …;
– Java collections
– java.util.Collections.List<Human> allHumans;
– java.util.Collections.Iterator<Human> I = allHumans.getIterator();
– Human h = i.next();
– And other collections/databases have similar concepts
16
Modern Internet Service Oriented Application Development –
Lecture 2: REST Details and Patterns
© Donald F. Ferguson, 2015. All rights reserved.
How do you Represent this in REST?
Well, There are a Few Ways
• Limit and Offset
– GET …/earth/humans?offset=0&limit=10
– GET …/earth/humans?offset=10&limit=10
– And this has a natural rendering in some languages, e.g. RDB
SELECT * FROM humans LIMIT 10 OFFSET 0;
• The collection is ordered by a key
– GET …/earth/humans?zipcode<=0&limit=10
– Look at the 10th entry in the result set, which has zipcode=“20135”
– And the next 10 are GET …/earth/humans?zipcode>”20135”&limit=10
• And you can combine all of this in freaky ways
– GET …/earth/humans?q=“hairColor=red&iq>100”&s=“weight,shoesize”&limit=10&offset=10
– Maps to a freaky SQL statement of the form
SQL SELECT * FROM humans WHERE hair_color=red AND iq>100
ORDERBY weight, shoe_size OFFSET 10 LIMIT 10
17
Modern Internet Service Oriented Application Development –
Lecture 2: REST Details and Patterns
© Donald F. Ferguson, 2015. All rights reserved.
This Pushes a lot on the Client
• A well-defined REST API hides this complexity using
– RFC 5988 Link headers
– The GET returns an HTTP header of the form
–
–
–
–
–
–
Links: <URL>; rel=“next”, <URL>; rel=“prev”, <URL>; rel=“first”, <URL> …
rel=“self” is the URL to the page you are on.
rel=“next” is the URL to the “next” page
rel=“prev” is the URL to the “previous” page
rel=“first” is the URL to the “first” page
rel=“last” is the URL to the “last” page
– GET on the URL with rel=“next” GETs the next page with a new set of next, prev, first, … related URLs
• This enables
– The client to specify a limit or the server to specifyc a limit
– And enable the client to iterate without know how the server is implementing iteration, e.g.
–
–
–
18
LIMIT and OFFSET
ORDER BY and values
……
Modern Internet Service Oriented Application Development –
Lecture 2: REST Details and Patterns
© Donald F. Ferguson, 2015. All rights reserved.
If You Do Not Like Headers
GET …/staff?status=active&limit=2 returns something like
• {
– “data” :
– [
– {“id” : 21, “manager” : {“link” : { “rel” : “manager”, “href” : “…/managers/9”}}},
– {“id” : 11, “manager” : {“link” : { “rel” : “manager”, “href” : “…/managers/101”}}}}
– ]
– “links” :
– [
– {“rel” : “next”, “href” : “…/staff?status=active&offset=2&limit=2},
– {“rel” : “first”, “href” : “…/staff?status=active&offset=0&limit=2},
– {“rel” : “last”, “href” : “…/staff?status=active&offset=210934&limit=2}
– ]
• }
19
Why did I highlight this?
Modern Internet Service Oriented Application Development –
Lecture 2: REST Details and Patterns
© Donald F. Ferguson, 2015. All rights reserved.
A subtlety
•
A GET on
–
–
•
Implicitly
–
–
•
You may not be the only one CRUD-ing humans
So freaky stuff can happen. For example, following the “self” link might not return the same JSON object that you just got.
The freaky stuff can happen in any database. This is why RDBs have isolation levels for cursors.
We cannot use isolation levels
–
–
–
20
REST is stateless and the server is not going to “keep the result” for “just for you”
So you have to tell the server how to re-compute the result
But,
–
–
•
•
Creates/computes a new collection resource “just for you”
That is the result of running the query or using the keys
But,
–
–
•
…/humans?iq=“100”
…/humans?q=“id=100&&hairColor=red”
Not really compliant with REST
Does not suit “Web scale” and “loosely coupled” applications
Which means we have to handle in other ways
Modern Internet Service Oriented Application Development –
Lecture 2: REST Details and Patterns
© Donald F. Ferguson, 2015. All rights reserved.