When IoT Meets Virtual
3D World
Liane Tarouco
Universidade Federal do Rio Grande do Sul
Brazil
Team
Liane Tarouco - PhD professor - chairman
Lisandro Granville – PhD professor
Leandro Bertholdo – Network engineer
Cesar Loureiro – Software engineer
Leandro Tibola – PhD student
Fabiana Sgobbi – PhD student
Fabricio Harpich – PhD student
Lucas Arbiza – MsC student
IoT and Virtual World
Internet of Things are becoming ubiquitous
and able to provide data to allow better
contextualization in virtual 3D world
This presentation report alternatives for
integrating IoT and VW
Using special designed midleware
Using derailleur based on SDN
Using direct access to web server
Using the silo approach provided by default in
IoT devices available in the market
Gardner – Hype Cycle 2015
Realism and engagement
Realistic environment promotes better engagement
Projects
AVATAR – Ambiente Virtual de Aprendizagem e
Trabalho Acadêmico Remoto
HIGIA - Habitat Individual e Guia Interativa de Atitudes
Virtual environment
AVATAR - Effectiveness of digital
laboratories as a learning resource.
HIGIA - Virtual 3D world to improve
motivation and self determination
Communication in and outside
virtual worlds
Communication with other objects in
virtual and/or physical worlds.
Virtual World being used
Servidor Open Simulator
Standalone
Server / grid
Clients
Firestorm
Singularity
Authoring in VW
1.
2.
Visual blocks
ScripTastic
Scratch for Open Sim
Flash Scratch to Linden Scripting Language
LSL & OSSL
• Notes
• Notepad++
• LSL Editor
Multmedia resources in VW
Presenting information
Slide presentation
Videos
Animations
Web pages
LSL comands for
interopreration
llHTTPRequest
Send data from Open Sim to na URL address
llOpenRemoteDataChannel();
llOwnerSay("Channel : " + (string) channel + "");
llRemoteDataReply(channel,NULL_KEY,"OK",1);
OpenSim sending SMS
12
LSL code for sending SMS
key myRequest;
integer CHANNEL = 0;
default
{
touch_start(integer number)
{
llSay(CHANNEL,“Type your phone number with area code 555191494040:");
llListen(CHANNEL, "", NULL_KEY, "");
}
listen(integer channel, string name, key id,
string message)
{
llSay(CHANNEL,“Sendin name: " +name + "=> test message sent to :" +message);
myRequest = llHTTPRequest("http://www.ieduca.com.br/enviasms.php",
[HTTP_METHOD, "POST",
HTTP_MIMETYPE, "application/x-www-form-urlencoded"],
"name="+name+"&celular="+message);
}
http_response(key request_id, integer status, list metadata, string body)
{
if (request_id == myRequest)
{
llSay(0,body);
}
}
}
Exchanging data with external
world
Virtual worlds can exchange data with learning
management systems, enterprise resource planning,
legacy systems, databases and Web systems, including
chatterbot.
These exchanges allow information to be sent and
received regarding simulations and the its users
User data can be collected to monitor user performance
in learning activities
Moodle
PHP/My SQL
Chatterbot
Virtual world
Real World
Applications to communicate with a virtual
world
an HTTP request to a Web service, where a prim
sends a request to the Web service, which
processes it and sends back an answer to OpenSim,
an HTTP script call that runs a remote script
program to manipulate data in a database
XML-RPC: enable two-way communication between
the virtual and physical world through remote
procedure calls and XML data transfer
Interoperation problems with IoT
Interoperation problems derived from
limited communication capabilities and low
computational power of IoT devices.
Most of smart devices, used in the context of Internet
of Things is not using generic standards interfaces.
One of the challenges in this context
derives from the need to integrate the
various devices and protocols making it
possible to obtain information about the
scenario being monitored
REMOA project
Environment Monitoring Network - project
targeting home solutions for homecare or/and
tele monitoring of patients with chronic
illnesses.
Health in family public program
17
Initial approach
Architecture
Data Collection
Agents
Middleware
18
Middleware
Wireless Access Point (AP) that supports
OpenWRT or DD-WRT.
Enables the development and deployment
of additional software, and supports
protocols
IPv4, IPv6, NAT
SNMP proxy
Gateway to other devices
19
Middleware issues perceived
Development of services features limited
by APs constraints
Complex deployment and maintenance
Complex mechanism to manage smart
devices
20
Refactoring
A solution to circumvent this problem may
derives from using a SDN based
approach, which turn an access point in a
flow switch.
This solution offers a possibility to divert
traffic to a remote server in order to
handle, transform and redirect data as
required to use it as input in virtual 3D
world.
SDN-based middleware refactoring
22
Architecture overview
23
VW integration with external
objects
Internet of Things
Sensors
Monitors
WIFI Camera with movement Sensor - WIFI
Pedometer - Bluetooth
IoT & Virtual Word
Camera captured images may be
presented as web pages used as
texture in 3D Objects
Internet of Things
Pedometer data recovered from FitBit
server and presented in VW
Conection with external system
link_message(integer sender_num, integer num, string msg,
key id)
{requestid=llHTTPRequest(“http://www.anyexternalsystem.com
/talkxml?id=”+id+&input=”+llEscapeURL(msg)+”&custid=”+cust,
[HTTP_METHOD ,"POST"], ””); }
http_response(key request_id, integer status, list metadata,
string body)
{ if (request_id == requestid)
if (msg == “/create”){
listen_to (id); }
HIGIA - academy
Prof. PhD. Liane Margarida Rockenbach Tarouco
[email protected]
Federal University of Rio Grande do Sul
Porto Alegre, Brazil