UT DALLAS
Erik Jonsson School of Engineering & Computer Science
StormRider: Harnessing “Storm”
for Social Networks
FEARLESS engineering
Social Network Applications
CHANGE IS A CONSTANT !!
How do you keep up with
the current trends and
important nodes in an
ever-changing network??
How do you
lookup previous
versions of a
network?
How do you
track changes
for a certain
node??
FEARLESS engineering
9/11 Terrorist Network
How do you track
what changes in his
neighborhood ??
FEARLESS engineering
Querying Evolving Networks
FEARLESS engineering
Motivation
• No current tools available for dynamic network
analysis
– Almost all current work focuses on a snapshot of a
network at a time
• No support for continuous queries on a dynamic
network
– No specific language to query a social network
• Can we build a cloud infrastructure to tackle these
two problems??
FEARLESS engineering
Architecture
Application
Model-SN Interface
Model-SN
Storm Interface
Add-Topology
Query-Topology
Analysis-Topology
Jena-HBase Interface
HBase View Interface
Storm
Storage Layer
Jena-HBase
Store
FEARLESS engineering
View Layer
HDFS
Files
HBase
Views
HDFS
Files
View Layer
Node Id Adj-List
Metric
Landmarks
DegC
CloseC
BetC
IsLandmark
Dist-To-ClosestLandmark
ClosestLandmark
sub1
sub2
sub3
0.75
0.5
0.5
Y
0
sub1
sub10
sub20
sub30
0.1
0.03
0.02
N
1
sub1
LandmarkId&NodeId
FEARLESS engineering
Nodes
Distance
Num-Of-Paths
Paths
sub1&sub10
2
1
sub1-sub2-sub10
sub1&sub50
2
2
sub1-sub5-sub50
sub1-sub25-sub50
Sample Add Topology for Twitter
Twitter-API-Spout
JenaHBase-Bolt
Storage
Node-centric View
Update-NodesView-Bolt
Count-UserDegree-Bolt
RankUsers-Bolt
MergeUsers-Bolt
Landmark-Selection
FEARLESS engineering
UpdateLandmarks-ViewBolt
Landmark-centric View
Sample Query Topologies for Twitter
I/P
•
O/p to
Client
Tracking neighborhoods: Use a topology that runs a continuous
query on RDF triples in Jena-HBase using a regular expression
SPARQL query
–
•
SPARQL-Query-Spout
SELECT ?x
WHERE { ex:John gleen:onPath( “[twitter:Has_Friend]/[twitter:Has_Friend]” ?x ) }
Version Management: Use the concept of RDF reification to add
version information for every triple added to the triple store. Then a
SPARQL query of the form:
–
SELECT ?z
WHERE
{ ?x twitter:Timestamp ?y .
?x rdf:subject <ex:John> .
?x rdf:predicate <twitter:Has_Friend> .
<ex:John> <twitter:Has_Friend> ?z .
FILTER( ?y <= “2012-01-25T21:00:00”^^xsd:dateTime ) }
can be used to find all friends of “John” before 9pm 01-25-2012
FEARLESS engineering
Sample Analyze Topologies for Twitter
Degree-C-Bolt
Twitter-User-Spout
Closeness-C-Bolt
Betweenness-C-Bolt
FEARLESS engineering