Integrating Active, Flexible and
Responsive Tertiary Prosumers into a
Smart Distribution Grid
WP3 INERTIA Local Control
and Automation Hub
Task 3.4 Multi-Sensorial Activity Flow
Detection & Modelling in controlled
environments
CERTH-ITI
INERTIA 4th Plenary Meeting – Stockholm - 9-10 September 2013
T3.4 Overview
Task 3.4 “Multi-Sensorial Activity Flow Detection and Modeling
in controlled environments
We are here
Deliverable
contributes to:
D3.3 Ambient User Interfaces, User
Behavioural Profiling and Activity Flow
Framework
(Prototype, Public, M24 September 2014)
Milestone
MS6 Final Version of the Occupancy Flow
Modeling and Prediction available
(M24 September 2014)
Involved Partners:
CERTH/ITI
ALMENDE
9-10 September 2013
11
2
INERTIA 4th Plenary Meeting, Stockholm
2
T3.4 Main achievements so far
• Explored different sensor clusters for a wide range of different types of
tertiary buildings, commercial building rooms and zones.
• Investigated algorithmic methods for extracting human motion and
occupancy flows in buildings based on the proposed Occupancy Modelling
and Prediction approach of Task 2.3.
• Initial prototype development in order to:
– examine and evaluate different algorithmic approaches
– define most appropriate methodology for the extraction and prediction of
occupancy flows
• Collected test data from Kinect cameras:
– installed in three monitored spaces
– used them as input on the prototype
– initial assessment of the implemented approaches
• Defined how privacy issues related to the sensors used will be handled
within INERTIA.
9-10 September 2013
INERTIA 4th Plenary Meeting, Stockholm
3
Prototype Development
• Development of a prototype to cover 3 functionalities
– Occupancy Detection and Extraction
– Occupancy and Flow Modelling
– Occupancy and Flow Prediction
• Modular design to allow easy modification
9-10 September 2013
INERTIA 4th Plenary Meeting, Stockholm
4
Occupancy Detection and
Extraction (1/3)
Target:
• correlate data from various sensors to extract number of
occupants per space/zone
• store them in an intermediate format and use them as
historical data
• detect individuals carrying RFIDs
Prototype Development:
• depth image processing algorithms to extract occupancy per
space/zone
• correlation of overlapped 3d-images in 3D-space (to overcome
unseen areas, obstacles etc)
• store data in an intermediate format – to give input for
occupancy modelling
• visualization of historical occupancy data
9-10 September 2013
INERTIA 4th Plenary Meeting, Stockholm
5
Occupancy Detection and
Extraction (2/3)
Prototype Testing:
• sample data collection
– installed Kinect cameras to
monitor 2 spaces at CERTHITI Building (Office and
Corridor)
– collected sample data for 9
days (7 days training set / 2
days testing set)
9-10 September 2013
INERTIA 4th Plenary Meeting, Stockholm
6
Occupancy Detection and
Extraction (3/3)
Next Efforts:
• extend monitored areas (based on Pilot audit of T1.3)
• collect sufficient amount of data
• check more sensors (e.g. CO2, PIR motion sensors)
• individual extraction (through RFIDs)
9-10 September 2013
INERTIA 4th Plenary Meeting, Stockholm
7
Occupancy and Flow
Modelling (1/2)
Target:
• create and update (maintain) building occupancy models (overall and
individual) according to task T2.3 based on historical data and typical
schedules
• use Open Reference Models and typical schedules for initialization
• implement various approaches and compare them
Prototype Development:
• Markov Chain Model (Overall approach) – Transition Matrices
– Parameters
 Transition Matrix Time Frame
 Start Time
 Observation Timestep
 Occupancy Type: exact number of occupants or occupancy ranges
(e.g. empty, low, etc.)
9-10 September 2013
INERTIA 4th Plenary Meeting, Stockholm
8
Occupancy and Flow
Modelling (2/2)
Prototype Testing:
• Transition Matrix Time Frame: 60min, 30min, 20min
• Observation Timestep: 1min
• Occupancy Type: exact number of occupants, occupancy
ranges (3, 4, 5 levels)
Next Efforts:
• address other model aspects (e.g. first arrival, last departure)
based on T2.3
• explore and implement update method
• consider Individual approach
• explore other modelling methods (e.g. HSMMs: Hidden semiMarkov Models)
9-10 September 2013
INERTIA 4th Plenary Meeting, Stockholm
9
Occupancy and Flow
Prediction (1/3)
Target:
• combine models with dynamic schedules and real-time
information in order to provide
– short-term prediction (few minutes after real-time)
– mid-term prediction (for the whole next day)
• give predicted occupancy per space
• give predicted location of occupants with RFIDs
• implement various algorithmic approaches and compare them
Prototype Development:
• prediction algorithm based on Markov Chain Model (Overall
approach)
9-10 September 2013
INERTIA 4th Plenary Meeting, Stockholm
10
Occupancy and Flow
Prediction (2/3)
Prediction Algorithm
• Inputs:
– model (stored Transition Matrices)
– real-time information: current state (number of occupants
per space)
– current time
– prediction time
• Outputs:
– predicted occupancy per specified timestep (only changes
depicted)
– visualization of the predicted occupancy flow
9-10 September 2013
INERTIA 4th Plenary Meeting, Stockholm
11
Occupancy and Flow
Prediction (3/3)
Prototype Testing:
• various testing scenarios
• current state, current time, prediction time entered manually
Next Efforts:
• incorporate dynamic scheduling information
• consider overlapping Transition Matrices
• consider how other aspects of the models (e.g. first arrival, last
departure) could be utilized based on T2.3
• examine Individual approach
• explore other algorithmic approaches (e.g. HSMMs: Hidden
semi-Markov Models)
9-10 September 2013
INERTIA 4th Plenary Meeting, Stockholm
12
Indicative Results:
short-term prediction
• Real-time occupancy extraction
(current state) has significant impact
on prediction result
Current
State: {4, 0}
Current
State: {2, 0}
• Transition Matrix Time Frame = 30min
• Observation Timestep = 1min
• Number of occupancy levels = 4
9-10 September 2013
INERTIA 4th Plenary Meeting, Stockholm
13
Indicative Results:
mid-term prediction
30min Transition Matrix works better here as it predicts the lunch break
• Transition Matrix Time Frame = 60min
• Observation Timestep = 1min
• Number of occupancy levels = 4
• Transition Matrix Time Frame = 30min
• Observation Timestep = 1min
• Number of occupancy levels = 4
9-10 September 2013
INERTIA 4th Plenary Meeting, Stockholm
14
Conclusions
• Implemented approach seems to give satisfactory
results
• Transition Matrix Time Frame appears to affect
prediction result (30min frame seems the best
solution so far) – Revise once more data are collected
• Using occupancy ranges gives better results than
using exact number of occupants
• More occupancy data are needed to have a more
clear view and be able to compare predicted with
actual values
9-10 September 2013
INERTIA 4th Plenary Meeting, Stockholm
15
Data Collection - Kitchen
Kinect
Camera
• Installed Kinect camera at the
kitchen of CERTH-ITI Building
and collected data for 9 days
• To divide kitchen into 2 zones
• To correlate kitchen data
with data collected from
the other two spaces
(Office, Corridor) and make
tests
9-10 September 2013
INERTIA 4th Plenary Meeting, Stockholm
16
T3.4 Next efforts
• Specify in detail the proposed combination of sensors needed
for various types of tertiary buildings along with their optimal
placement based on use-cases and system requirements
• Continue prototype development, extending functionalities
and implementing more algorithmic approaches (e.g. HSMM:
Hidden semi-Markov Model), in order to cover all different
aspects identified
• Acquire more occupancy data collected from CERTH or other
projects (e.g. Adapt4EE), towards better assessment of the
different approaches
• Take into account scheduling information
9-10 September 2013
INERTIA 4th Plenary Meeting, Stockholm
17