Data on the Web
Exchange, Formats and State
Advanced Web-based Systems | Misbhauddin
Components of Data Exchange
Two applications want to share data over the Internet
• Initiating Action: Things don’t happen without any reason. A triggering
mechanism is needed.
• Automatic: started at a fixed time (chron job, feed exchange)
• User Initiated: form submission
• Data Format: Data needs to be stored and transferred in a format understood by
both applications.
• Data Transfer Mechanism: Process to transfer the data over the internet
Advanced Web-based Systems | Misbhauddin
Initiating Action
• Human Initiated
• Import or export a file
• Running a command
• Scheduler
• Relying on a fixed schedule
• Checking information everyday (weather), every second (traffic update) or
any other period of time. Custom programmed or OS based (chron)
• End User Action
• Form Submission
Advanced Web-based Systems | Misbhauddin
Data Format
• Common format that is understood by the communicating
applications
• Many formats have been seen over the years
• CSV
• XML
• JSON
• HTML5 Data
Advanced Web-based Systems | Misbhauddin
Data Formats
• Why not use existing DBMS
• Very popular and provide excellent data storage and access
mechanisms
• Main reason for not using them - Incompatibility
• Formats
• Internal representations
• Data types
• Encoding
Advanced Web-based Systems | Misbhauddin
CSV – Comma Separated Values
•
•
•
•
•
Fixed-text format
Values separated by a delimiter – usually a comma (,)
Easier to extract data
Simplest of all the data formats
Easy for the customer to import – can use excel as an alternative
to read the contents
• Commonly used to export and import data for variety of web
applications
Advanced Web-based Systems | Misbhauddin
CSV – Comma Separated Values
• Program receiving the data is told about the length and datatype
of each field.
“Susan”,”f”,25
“Mark”,”m”,37
• Challenges
• What if the data has the delimiter inside the text itself
• Data is fixed
• Needs additional programming
• CSV is standard for data export.
Advanced Web-based Systems | Misbhauddin
CSV – Language Support
• Client-Side
• No direct JavaScript Functions
• JQuery has many functions that can read and write CSV files
easily
•
•
https://code.google.com/p/jquery-csv/
http://papaparse.com/ [CSV ↔JSON]
• Server-Side
• Many server-side PLs provide functions to handle CSV files
• Example – PHP
•
•
fputscsv(file-handle, data) is used to store data into a CSV file
str_getcsv(data) is used to parse CSV data
Advanced Web-based Systems | Misbhauddin
XML – eXtensible Markup Language
• Popularly used to exchange data over the Internet
• Create data structures that can be exchanged between
incompatible systems
• Tag / Element based language
• Derived from SGML
• Resembles HTML but more stricter
• Customizable Tags
• No SET Element Names – “Self-Describing”
• Interoperable
• Machine-readable
Advanced Web-based Systems | Misbhauddin
XML – eXtensible Markup Language
USES
Data
Presentation (xhtml)
<contact>
<name>Mohammed Misbhauddin</name>
<office>CCSIT 2078</office>
<city>Al-Ahsa</city>
<province>Eastern</province>
<zip>32168</zip>
<phone type="office">03-589-9253</phone>
<email>[email protected]</email>
</contact>
<table>
<tr>
<th>Name</th>
<td>Mohammed Misbhauddin</td>
</tr>
…………………
………………..
………………..
</table>
Advanced Web-based Systems | Misbhauddin
Name
Mohammed Misbhauddin
Office
CCSIT 2078
Phone
03-589-9253
Email
[email protected]
XML for Structuring Data
<contact> Data
<name>Mohammed Misbhauddin</name>
<office>CCSIT 2078</office>
<city>Al-Ahsa</city>
<province>Eastern</province>
<zip>32168</zip>
<phone type="office">03-589-9253</phone>
<email>[email protected]</email>
</contact>
Attributes
<phone type="office">03-589-9253</phone>
Meta Data
Data
That describes the data
Advanced Web-based Systems | Misbhauddin
XML – Applications
• Used as a source for many application communication formats
• Universal Business Language (UBL)
• Universal Plug ‘n’ Play (UPnP)
• Word Processing
• Open Data Format (ODF)
• Open Office XML (OOXML)
• Scalable Vector Graphics (SVG)
• RSS Feeds
Advanced Web-based Systems | Misbhauddin
XML – Applications
Weather Data Communication
Posts a weather feed
<!-- The Weather Channel Interactive, Inc. All Rights Reserved. -->
<weather ver="2.0">
<loc id="SAXX0031">
<dnam>Al Ahsa, Saudi Arabia</dnam>
<tm>01:00 PM</tm>
<lat>42.38</lat>
<lon>-71.13</lon>
</loc>
<cc>
<temp>
<value>88</value>
<deg>F</deg>
</temp>
……….
</weather>
Advanced Web-based Systems | Misbhauddin
XML – Applications
Maps: Google Maps & Google Earth
Keyhole Markup Language (KML) is an XML notation for expressing geographic annotation and
visualization within Internet-based, two-dimensional maps
Location Data
<kml xmlns="http://schemas.opengis.net/kml/2.2.0/ogckml22.xsd">
<Style id="style1">
<IconStyle>
<Icon>
<href>http://maps.google.com/mapfiles/ms/micons/green-arrow.png</href>
</Icon>
</IconStyle>
</Style>
<Placemark>
<name>CCSIT</name>
<styleUrl>#style1</styleUrl>
<Point>
<coordinates>25.334693,49.600021</coordinates>
</Point>
</Placemark>
……….
</kml>
Advanced Web-based Systems | Misbhauddin
XML – Applications
News Feeds
Syndication Feeds: RSS
<rss xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns# version="2.0">
<channel>
<title>Saudi Gazette - Kingdom</title>
<link>http://www.saudigazette.com.sa</link>
<description>Saudi Gazette Newspaper</description>
<pubDate>Sat, 02 Mar 2013 22:05:34 GMT</pubDate>
<item>
<title>Crackdown to flush out infiltrators continues</title>
<link>http://www.saudigazette.com.sa/index.cfm</link>
<description>
<img src='http://www.saudigazette.com.sa/ki09-big.jpg'
Falih Al-Dhibyani and Abdullah Gharman Okaz/Saudi Gazette
RIYADH/ABHA &amp;mdash; No murders, thefts, or attacks against
</description>
<pubDate>Sat, 02 Mar 2013 21:00:00 GMT</pubDate>
</item>
……………
</rss>
Advanced Web-based Systems | Misbhauddin
XML Meta Data
• XML allows users to define their own tags
• Tags describe the data that it contains (meta-data)
• Links to document that describes the data is stored on a server
and linked in the XML document
• Documents are called XML Schema (.xsd)
• Included in the xmlns attribute of the root element
• xmlns: XML Namespace
• Example - KML
<kml xmlns=" http://schemas.opengis.net/kml/2.2.0/ogckml22.xsd ">
Advanced Web-based Systems | Misbhauddin
XML Rules
• Well-formed XML Documents
•
•
•
•
•
Must have one and only one root element
Elements must be properly nested
Attributes must be “quoted”
XML elements are case sensitive
End tags are required
•
Empty elements can contain the end marker
• Attributes cannot be repeated
Advanced Web-based Systems | Misbhauddin
XML DOM Model
DOM Tree (Document Object Model)
Advanced Web-based Systems | Misbhauddin
XML DOM Model
parentNode
Professor
firstChild
In a node tree, the top node is called the root
Every node, except the root, has exactly one parent node
A node can have any number of children
A leaf is a node with no children
Siblings are nodes with the same parent
nextSibling
id
childNodes
•
•
•
•
•
name
previousSibling
rank
lastChild
dept
Advanced Web-based Systems | Misbhauddin
XML Processing
• Step 1: Load the file into the DOM
• All major browsers have a built-in XML parser to read and
manipulate XML
• XML parser converts XML into an XML DOM object
• How
•
•
•
Create an XMLHttpRequest object
Use the open() and send() methods of the XMLHttpRequest object to
send a request to a server
Get the response data as XML data
https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest
Advanced Web-based Systems | Misbhauddin
XML DOM Properties
• These are some typical DOM properties:
• x.nodeName - the name of x
• x.nodeValue - the value of x
• x.parentNode - the parent node of x
• x.childNodes - the child nodes of x
• x.attributes - the attributes nodes of x
Note: In the list above, x is a node object.
Advanced Web-based Systems | Misbhauddin
XML DOM Methods
• Methods are what you can do
• x.getElementsByTagName(name) - get all elements with a
specified tag name
• x.appendChild(node) - insert a child node to x
• x.removeChild(node) - remove a child node from x
Note: In the list above, x is a node object.
Advanced Web-based Systems | Misbhauddin
XML Processing
• Step 2: Access elements of the DOM Tree
• You can access a node in three ways:
1. By using the getElementsByTagName() method
2. By looping through (traversing) the nodes tree.
3. By navigating the node tree, using the node relationships.
Advanced Web-based Systems | Misbhauddin
• nodeName
• nodeValue
• nodeType
XML Node Types
nodeType
Name
nodeName
Description
nodeValue
1
Element
element Name
Represents an
element
Null
2
Attribute
attribute name
Represents an
attribute
attribute value
#text
Represents textual
content in an element content of node
or attribute
3
Text
name
type
full
#text
Mohammed Misbhauddin
Advanced Web-based Systems | Misbhauddin
XML Node Lists
• Child Nodes
• x.childNodes;
• Returns an array with the nodes of the parent x
• Attribute Nodes
• y = x.attributes;
• Returns a Node Value Map
• Accessed using
• y.getNamedItem(“type")
Advanced Web-based Systems | Misbhauddin
XML White Spaces
• XML often contains new line, or white space characters, between
nodes.
• This is often the case when the document is edited by a simple
editor like Notepad.
• Always check the type of the node as the total nodes can be
deceptive.
Advanced Web-based Systems | Misbhauddin
XML Technologies
• Specification
• Data Type Definition (DTD)
• Namespaces
• Styling XML
• XSLT – Transform XML to HTML and other formats
• xPath – Target XML content
• XSL-FO – Format XML to different media
Advanced Web-based Systems | Misbhauddin
JSON – JavaScript Object Notation
• Similar to XML, designed to be human-readable
• Uses the JavaScript Notation
• But is truly platform-independent
• Text-based Code to create Objects
Advanced Web-based Systems | Misbhauddin
JSON – JavaScript Object Notation
• Used for serializing & transmitting structured data over network
connection
• Primarily used to transmit data between server and web
application
• Web Services and API.s use JSON format to provide public data
• Supported by most modern programming languages
• Positives
•
•
•
Easy to read and write JSON
Lightweight text based interchange format
Language independent
Advanced Web-based Systems | Misbhauddin
JSON – Syntax
• Data is in name/value pairs
• Separated by Comma (,)
• Curly brackets are for objects
• Square brackets are for arrays
• Names are always strings
• Values = strings, numbers, Boolean, arrays, JSON objects or null
Advanced Web-based Systems | Misbhauddin
JSON – Sample Object
{
"book": [
{
"id":"01",
"language": "Java",
"edition": "third",
"author": "Herbert Schildt"
},
{
"id":"07",
"language": "C++",
"edition": "second"
"author": "E.Balagurusamy"
}
]
}
Advanced Web-based Systems | Misbhauddin
JSON – Creation in JS
var books = { "Pascal" : [
{ "Name" : "Pascal Made Simple", "price" : 700 },
{ "Name" : "Guide to Pascal", "price" : 400 }
],
"Scala" : [
{ "Name" : "Scala for the Impatient", "price" : 1000 },
{ "Name" : "Scala in Depth", "price" : 1300 }
]
}
Advanced Web-based Systems | Misbhauddin
JSON Parsing
• Use eval() to parse and convert JSON to JS Object
• Another operation which is more convenient
• JSON.parse()
• Convert JS Object to JSON
• JSON.stringify();
Advanced Web-based Systems | Misbhauddin
JSON vs. XML
Factor
JSON
XML
Size
JSON is light-weight and
easy to read
XML is verbose. Lots of
information
Array Usage
JSON allows arrays
XML is purely structured
data. So no arrays.
Parsing
Simple. Use eval() to
convert to a JS Object
A number of steps to parse
and retrieve the data
Advanced Web-based Systems | Misbhauddin
JSON Tools
• JSON Lint
•
Validator for JSON Object
• http://jsonlint.com/
• JSON Online Editor
•
•
Every browser provides the user the ability to define his own set of styles.
http://jsoneditoronline.org/
• XML to JSON Converter
•
•
Converts XML Data to JSON
http://www.utilities-online.info/xmltojson
Advanced Web-based Systems | Misbhauddin
HTML5
• Custom Data Attributes using data-*
•
Application Data
• Can be read and written from JavaScript
• Using setAttribute and getAttribute
<div data-something=“new” id=“test”>
<script>
var a = document.getElementById(“test”);
var attr = a.getAttribute(“data-something”);
//do Something with the value
</script>
Advanced Web-based Systems | Misbhauddin
HTML5
• Using DataList
•
•
Datalist holds values for autocomplete options
Not supported by iOS
<datalist id="browsers">
<option value="Internet Explorer">
<option value="Firefox">
<option value="Chrome">
<option value="Opera">
<option value="Safari">
</datalist>
<input list=“browsers”/>
Advanced Web-based Systems | Misbhauddin
Data for State
• The HTTP protocol is stateless
•
Means that the state of a Request-Response transaction is not stored
Gmail Server
Client
Username=“Mohammed” && Password= “*********”
success
getMain()
error
Advanced Web-based Systems | Misbhauddin
Data for State
• State is maintained externally
• Needs server-side coding to create variables
State Recognition
Cookies
Sessions
Returned and stored in the
user's browser
Session data is stored on your web server
(often stored in a public temporary
directory on the server)
Life span of a cookie can be
set to almost any duration
Advanced Web-based Systems | Misbhauddin
Sessions have a
predetermined short life
(depends on config)
Cookies
Store Cookie Data
Gmail Server
Username=“Mohammed” && Password= “*********”
Success , Cookie Data
Client
getMail(), cookie
PHP Function for set and getting a cookie value
setcookie (name, value, expire)
Set Cookie
$_COOKIE[name]
Get Cookie Value
Advanced Web-based Systems | Misbhauddin
setcookie (name, “”, currenttime-1)
Delete Cookie Value
Sessions
Store Session ID
Store Session Data
Username=“Mohammed” && Password= “*********”
Success , Session ID
Client
getMail(), Session ID
Gmail Server
PHP Function for set and getting a Session value
session_start();
$_SESSION[name] = value;
Set Session
$_SESSION[name]
Get Session
unset($_SESSION[name]);
session_destroy();
Delete Session
Advanced Web-based Systems | Misbhauddin
Data for State
• HTML5 provides mechanisms known as Local Storage
• Store and retrieve data locally
• Add state to web without server-side coding
• Uses two functions only
SET ITEM: localStorage.setItem(“ItemName”,”ItemValue”);
GET ITEM: localStorage.getItem(“ItemName”);
Advanced Web-based Systems | Misbhauddin
Data Transfer Mechanism
COMING SOON
Simple Object Access Protocol (SOAP)
Web Services Description Language or WSDL
Representational State Transfer (REST)
Advanced Web-based Systems | Misbhauddin
Why not change the DB
If the Data exchanged over the internet is not Relational, then why Relational DBMS
This is why we have schema-less databases
COMING SOON
Relational vs. Schema-less Databases
Advanced Web-based Systems | Misbhauddin