Streaming IoT Analytics
with the PI Integrator for
Azure, Cortana Intelligence
and Azure IoT
Presented by Prabal Acharyya
Landry Khounlavong
USERS CONFERENCE 2017
@ osisoft
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC
USERS CONFERENCE 2017
@ osisoft
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC
IoT Analytics Challenges
USERS CONFERENCE 2017
@ osisoft
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC
3
Transforming the Value of Data
1 | Data
2 | Information
3 | Intelligence
Real-time Decision
Support
Business/Operation
Intelligence
USERS CONFERENCE 2017
@ osisoft
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC
Microsoft big data & advanced analytics
Solutions
Intelligence
Cognitive Services, Bot Framework. Cortana
Dashboards and Visualizations
Power BI
SSRS, DataZen, R
R visualizations
Machine learning and Analytics
Machine learning, Stream Analytics, HDInsight
SSAS, SQL Server R Services
Microsoft R
Big Data Store
Data Lake, SQL, DW
OLTP, DW, Hadoop, EDSs
Hadoop, Teradata, Linux, Windows
Information Management
Data Factory, Data Catalog, Event Hubs
SSIS, DQS, MDS
Spark SQL/ETL
Cortana Intelligence Suite
USERS CONFERENCE 2017
SQL Server 2016
@ osisoft
Microsoft R
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC
Modeling
Preparation
Ingest
Transform
Cleanse
Denormalize
Profile
Explore
Visualize
Feature &
Algorithm
Selection
Model
Testing &
Validation
Operationalization
Models
Visualizations
Deploy Apps,
Services &
Visualizations
Act:
Score,
Visualize
Measure
USERS CONFERENCE 2017
@ osisoft
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC
Data Scientists perform too much Data janitor-work
USERS CONFERENCE 2017
@ osisoft
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC
9
OSIsoft and Microsoft: Joint Value
USERS CONFERENCE 2017
@ osisoft
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC
Observed advance analytics goals
• Augment decision making with data
• Minimize work needed to begin analyzing
• Invest in out-of-the-box, reusable capabilities for
advanced analytics
• Prove advanced analytics can achieve a business
outcome
USERS CONFERENCE 2017
@ osisoft
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC
11
Challenge: making sense of the data
Solution:
QI-122
80-5.Net Volume DC.SJ.PUE
GE01_A_DT
TI-102
DC.Zero
DC.SJ.C1.Z3.R3.PDU1.PF
DY-108
GE01_A_DT
trigger
AC09.Power
GE05_Energy C1:14AT5 AC03.Air Flow
FeedBin.Cmt Boiler Cold Reheat Pressure
GE04_OSAsset1.Problem
DC.SJ.SiteRealTim Anacortes Refinery.Alkylation.Asset
B737_FG117 DC.TimeLoad
eITLoad.PR FT9001
FI-151 403511195_Wind Speed
s
Problems B210_FG005.KPIExcursion D-110.Tank Pressure.PV
bf5e1d1d-39c9FI-101
02F102.1HRAVG BGT001 AQUA2-SI005.PV
GE04_DT QI-121 GE03_V_WIN
AC04.Air
Flow
4-36.NetVolume
0_ENG_M ODE _STS
PI-111 facility_output
02T100 03LBB02CT001-2 4b5b-b3d3-c2ce05fa3a26
DC.Rk07R
DM-05:BW.R
PI System
GE02_Energy
FI-151
AQUA2-TI-201.PV
02T103.Q DC.Z1R
AlarmTest.Input.Float32.1
FI-101
DC.NY.Actual.PWR.day.Tot
0_CM P_ H D R _SU C _ PRS
AlarmTest.Input.Float32.10
364511575-AC Power
BGE003 FI-111 02T100
AF_FLOW3 02:T103
AQUA2-SI005.PV GE02_Energy
AT401
0_ENG_MODE_STS
ACEDemo.Unit1.Output
TI-178 B352_W778
02F104 CD:F161
94:BW.R TI-101
FI-151
0_CMP_SVLV_PCT DC.Z1R
02T103.Q
0_CMP_HDR
80-13.Net
1-13.Net
F723_E889
Volume
369512185-Temp
B045_FG978
DY-131
DC.SJ.PUE
Compressor-439.Feed
fic1001.C GE02_OT
Rate
bf5e1d1dFI-101
DC.CH.DCE FIC39c9- 4b5b-b3d3-
172:210
GE01_DT
FI-121 AF_FLOW3
0_ENG_MODE_STS
DY-131:166
c2ce05fa3a26
AT401
DM-05:BW.R
0_CLR_FINAL_OUT_B
F506_E990
_TMP
339511775_Clear Sky Global
Horiz
GE01_DT
GE03_Q
GE01_TD
FI-101
Boiler-334.Feed Rate Volume
02F102.1HR BGE003 FI-111 02T100
AVG
AF_FLOW3 02:T103
BGT001
PI-111
ACEDemo.Unit1.Output
TI-178 B352_W778
facility_output
0_CMP_SVLV_PCT
02F104 CD:F161
AT401
QI-109
GE01_DT 409510395_Wind Speed
GE01_DT Cooling Fan-711.Feed Rate
bf5e1d1d-39c9- 4b5b-b3d3-c2ce05f
Boiler-209.Fuel Gas
DC.Srv01R
Boiler- AlarmTest.Input.Float32.1 Flow DC.Srv01R
94:GRDIDX.Tr
125.Fuel Gas
Volume FI-151
D-110.Tank
DM-05:BW.R 0_ENG_AUX_STS
DC.SJ.C1.Z1.R1.Rk06.S2.
O03.PWR
QI-111FinalProductBin.OnTIC-181
94:GRDIDX.ProdID
Boiler-209.Fuel
Gas Flow
AF_NOISE
Pressure.P
V Boiler Feed
GE04_Status
02F100 fasttag 02:F101.C
AQUA2-SI005.PV GE02_Energy
02T103.Q
GE05_ES T
fic1001.C FR5001
FT9001
AF_NOISE
DM-05:BW.R
Weather
Conditions
Relative Humidity: 34%
Current Temp: 85 F
High: 92
Low: 57 F
Wind: 8 mph/N
Crude Furnace
FT9001 TI-145
fasttag FR2001
FeedBin.Cmt
Draft Pressure: -0.5 WC
Stack Temp: 316 F
Oxygen: 2.5%
Outlet Temp: 840 F
Cold Oil Velocity: 6 ft/sec
TI-178
GE04_OS
403511195_Wind
DC.Zone1.Number
Speed
DailyTrigger FrqPrbCost_ER FT9001
FT9001
45-2.Net Volume
DC.C2Z1.Pwr.Rippl FR5001
AF_NOISE
FT9001
GE01_A_DT
GE01_CON e
DC.SJ.SiteRealTim
eITLoad.PR FT9001
409510395_Wind Speed
QI-109 GE01_DT
Cooling Fan
DM-05:BW.R
DC.SJ.C1.Z1.R1.R
k06.S2.O03.PWR
T I-178
GE01_A_DT
fic1001.C
Active Meters
GE01_A_DT aso
PI-115
Operating Pressure: 110 psi
Charge Rate: 14 gph
Mix Valve Pressure: 8 psi
Water Rate: 8%
DC.SJ.ITLoad.P
WR
GE02_Energy
GE01_DT
asset1_output
DC.Z1R 0_CMP_HDR _SUC_PRS
FI-111
1-8.Net
Volume
1-16.Net Volume
03LBA32CT0
CB1992_MS
01-2
0_CMP_FLOW_TOTAL
0_ENG_MODE_STS
FI-151
Pump #1
AT401
% CO2
AlarmTest.Input.
Float32.1
FIC-144
Crude Desalter
AT401
0_CLR_FINAL_OUT_B_TDC.Srv06R GE04_Energy
MP
F506_E990
TI-121 FT9001
339511775_Clear
Sky
Global
Horiz
GE01_DT FAC.OAK.Power-Kh-Val.PV
DY-101
02:F101.C
QI-111
94:GRDIDX.Pr
odID Boiler-
FinalProductBin.On
FR2001 TIC-121 209.Fuel Gas
02:F101.C
Aso AT401 DC.Srv01R
USERS CONFERENCE 2017
Flow
@ osisoft
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC
12
PI System stores and organizes data
PI Data
Archive
Element
Attributes
PI Asset
Framework
PI Event
Frames
Event Frame
Attributes
Process Excursion
Shift
Startup
Shutdown
Environmental Excursion
USERS CONFERENCE 2017
@ osisoft
Downtime
Batch
Grade Change
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC
13
Challenge: cleaning & preparing sensor-based data
Solution:
PI Integrator for Microsoft Azure
Spike /
Out of Range
Oil
Temp
Additional
Measure
03:00
03:10
03:20
03:30
03:40
USERS CONFERENCE 2017
Oil Temp
Speed
Speed
Uneven Spacing
Oil Temp
Oil
Temp
Torque
Torque
Speed
Comm Failure
Different
start times
Speed
03:00
9.8
25
50
03:10
10
24
50
10.0
60
03:20
10
25
50
10.2
60
9.8
60
03:30
03:40
9.9
25
50
10.0
120
03:50
9.9
25
45
10.1
60
03:50
@ osisoft
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC
Challenge: implementing data science experiments
Solution: Cortana Intelligence Suite
Quick deployment
of resources
Off-the-shelf
data science tools
USERS CONFERENCE 2017
@ osisoft
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC
Leveraging the PI System and Cortana Intelligence
to Increase Process Efficiency
COMPANY and GOAL
Picture / Image
Deschutes Brewery is the 7th largest craft brewery in
US, and wanted to maximize production with its
existing infrastructure to fund construction of a 2nd
brewery in Roanoke, VA
CHALLENGE
Company
Logo
SOLUTION
Batch’s phase transition
happens between manual
density measurements
occurring every 8-10 hours
Use data science to achieve
accurate predictive analytics
for determining a batch’s
density measurements
• Impact: Losing up to 72 hours in
production time
•
•
•
•
•
PI System
PI Integrator for Microsoft Azure
SQL Data Warehouse
Azure Machine Learning
Azure Data Factory
USERS CONFERENCE 2017
@ osisoft
RESULTS
Ability to eliminate
production time losses and
increase production
capacity
• Accurate predictions of when a
batch’s phase transitions from
fermentation to free rise
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC
16
An example of how to operationalize predictions
DATA
SOURCES
PI System
PREPARE
PI Integrator for
Microsoft Azure
INGEST
ANALYZE
SQL
Data Warehouse
Machine
Learning
PUBLISH
CONSUME
SQL
Data Warehouse
Cortana
Power BI
Predictions as Future Data
Azure Data Factory (Orchestration)
On Premises
Web/LOB
Dashboards
On Cloud
USERS CONFERENCE 2017
@ osisoft
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC
17
Bridging the PI System and Microsoft Azure
PI Integrator for
Microsoft Azure
Released
Nov 2016
SQL Database
SQL Data Warehouse
Data Lake Store
Summer 2017
Event Hub
IoT Hub
USERS CONFERENCE 2017
@ osisoft
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC
18
Announcing
Red Carpet Incubation Program (RCIP)
a limited service offering
alongside Microsoft
that uses a process
to accelerate advanced analytics projects
for selected customers
USERS CONFERENCE 2017
@ osisoft
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC
Call to Action &
Summary
USERS CONFERENCE 2017
@ osisoft
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC
21
Analytics Journey => Transforming the Value of Data
1 | Data
Basic Analytics
2 | Information
3 | Intelligence
Descriptive &
Diagnostic Analytics
Predictive &
Prescriptive Analytics
Boiler-209.Fuel Gas Flow DC.Srv01R 94:GRDIDX.Tr igger AC09.Power
GE01_DT
409510395_Wind Speed
QI-109 GE01_DT
Cooling Fan-711.Feed Rate
DC.SJ.C1.Z1.R1.Rk06.S2.O03.PWR QI-111 FinalProductBin.On
94:GRDIDX.ProdID Boiler-209.Fuel Gas Flow fic1001.C FR500118.Net Volume Coal Motor Load 02F100.TOT.EV 3LBA32CT001-2
Weather Conditions
DM-05:BW.R
DC.SJ.PUE
TI-102
DC.Zero
DY-108 DC.SJ.C1.Z3.R3.PDU1.PF GE01_A_DT
403511195_Wind Speed DC.Srv01R Boiler-125.Fuel Gas Volume
1-16.Net Volume CB1992_MS 0_CMP_FLOW_TOTAL GE02_Energy
Anacortes Refinery.Alkylation.Asset Problems B210_FG005.KPIExcursion
AlarmTest.Input.Float32.1
FT9001
FI-101
AQUA2-TI-201.PV
DM-05:BW .R
Boiler Feed Pump #1
FIC-144 02F100 fasttag
FI-151
DC.C2Z1.Pwr.Ripple
0_ENG _ AUX_STS
02F102.1HRAVG BGT001
GE05_Energy
B737_FG117 DC.TimeLoad
D-110.Tank Pressure.PV
DC.Rk07R
GE04_Energy
GE03_V_WIN
DC.Srv06R
TI-121
FeedBin.Cmt
DailyTrigger
FrqPrbCost_ER
facility_output
Coal Motor Load
02F100.TOT.EV
03LBA32CT001-2
DC.SJ.ITLoad.PWR
TI-145 FR2001
TI-178
GE04_OS
F T 9 0 01
FT9001
FR5 0 01
AF_NOISE
FT9001
FAC.OAK.Power-Kh-Val.PV
DC.Zone1.Number
PI-111
FeedBin.Cmt
Boiler Cold Reheat Pressure
QI-121
Wind: 8 mph/N
AT401
FT9001
Energy Efficiency
GE01_ A_ DT
FI-111
C1:14AT5 1-8.Net Volume
AC03.Air Flow
GE04_DT
Low: 57 °F
339511775_Clear Sky Global Horiz GE01_DT
D-110.Tank Pressure.PV
GE01_CON AlarmTest.Input.Float32.1
High: 92 °F
DC.SJ.SiteRealTimeITLoad.PWR
bf5e1d1d-39c9-4b5b-b3d3-c2ce05fa3a26
0_CLR_FINAL_OUT_B_TMP F506_E990
Relative Humidity: 34%
Current Temp: 85 °F
DY-131
Downtime
Crude Furnace
Total Production
Draft Pressure: -0.5 WC
Stack Temp: 316 °F
Oxygen: 2.5%
Firebox Temp: 860 °F
Outlet Temp: 840 °F
Cold Oil Velocity: 6 ft/sec
$
DC.SJ.PUE
fic1001.C
FT9001
GE02_OT
GE01_DT
fasttag
AT401
USERS CONFERENCE 2017
@ osisoft
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC
Need OT Context – “must” for Analytics & IoT
Operational Context
High Fidelity Historical….. and Future Data from Analytics
Time-series data
•
•
•
Temperature
Pressure
Flow
Element
Attributes
Asset details
•
•
•
Name
Model
Manufacturer
Process
•
•
Well drilling
Site
USERS CONFERENCE 2017
@ osisoft
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC
23
Asking How & Why: PI Event Frames easily identifies events
for deeper root cause analysis
• The most powerful way to automatically bookmark PI
time-series data
• More meaningful use for engineers and business users
• Easier to find, analyze, and report on
Downtime
Excursion
Batch
Related Data
USERS CONFERENCE 2017
Event Attribute Value
Name
Ex 20141215-0002
Start
15-Dec-2014 10:35:02
End
15-Dec-2014 10:47:26
Duration
12 min, 24 sec
Asset
Boiler-352
Excursion Type
High Violation
Fuel Gas
Flow.Avg
37.12 k sft3/h
Fuel.Start
823.48 k sft3/ton
myPIKPI.Max
47.19 bbl/d
@ osisoft
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC
7
Streaming and Batch – IoT Analytics
Summer 2017
Event Hub
IoT Hub
Released
Nov 2016
PI Integrator for
Microsoft Azure
USERS CONFERENCE 2017
SQL Database
SQL Data Warehouse
Data Lake Store
@ osisoft
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC
USERS CONFERENCE 2017
@ osisoft
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC
USERS CONFERENCE 2017
@ osisoft
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC
USERS CONFERENCE 2017
@ osisoft
#OSIsoftUC
© Copyright 2017 OSIsoft, LLC