DevOps – Test Automation for IOTs
Santosh Vaswani – Technical Manager
Govind Soni – Member of Technical Staff
eInfochips Ltd.
Abstract
IoT – Internet of Things is the growing market. With this there is a need of
testing IoT Devices in integration with different Software Components,
Firmware and Hardware. In this presentation we are going to talk about:
• DevOps continuous testing with end-to-end test scenarios in
web/mobile and devices
• Challenges in continuous testing of IoT Devices as part of DevOps and
techniques to overcome these challenges.
2
Connected Devices
3
Connected Devices Testing – Business Challenges
•
Agile enterprises require a software
test environment for continuous
testing.
•
IoT device-makers work with sensors,
data storage, electro-mechanical
parts and microprocessors.
•
Enterprise software projects require
faster response time to real world
problems.
In order to bring the product to market
faster, it is required that they are able to
test each and every element in real time.
Business teams have three options for addressing
inadequate automation:
• Invest heavily in manual tests
• Delay the release
• Give up
Customer’s major challenges in involving
Vendors:
1. Risk of not getting successful results
2. Slow automation due to tightly coupled,
complex automation framework unable to
support devices
3. Inability to provide end-to-end product
automation (software, firmware and devices)
4
DevOps in Product Engineering
IT DevOps and Product Engineering DevOps differences:
1. Devices
2. Multiple Product Development Pipelines
3. Different Process, Different Skills, Different Teams in each Development
5
DevOps in Product Engineering Line
6
DevOps Challenges in Continuous Testing
Complex and fragmented Dev Pipeline: With multiple application delivery end points, including web, mobile, and
device(s) the team composition and work flow are more challenging to streamline than pure play IT APPs.
Environment as a Code: Product engineering space brings additional challenges of virtualizing multiple associated
devices along with server infra. The concept of ‘Infrastructure as code’ needs to be extended towards offering
complete ‘Environment as code’.
Product Pipeline: More variation on product delivery to multiple customer (customization & enhancement) and
market segment (Low, mid, & high). This presents varied production environment difficult to reproduce and
maintain at Dev stage.
Legacy Device & Solution Support: Product along with cloud infrastructure brings requirement for
managing, updating, and maintaining existing devices on field along with new added, again increasing
variations and complexity for Devops.
Release cadence: Different component of solution including firmware, web app, mobile app, & pc app have
different release cadence making unified release plan a challenge
7
•Virtualization/Simulati
on of test environment
to enabled continuous
testing
•Automation execution
with simulation and
physical devices
•Performance Testing
with network
virtualization
•Integration
Automation
framework with CI tool
•Parallel Remote
automation execution
8
•Auto Build
Deployment on
Devices and Servers
•Manage Rollback
deployment on live
environment
•Automated Alerts on
failure scenarios and
performance issues
•Generate Business
Analytical reports for
future reference
Continuous Monitoring
Continuous Delivery
•Integration of multiple
pipelines (Device, Web
and mobile) and
prepare main and
customized builds
•Automate and
Configure Environment
setup
•Trigger automated
code review/unit
tests/regression tests
•Generate and publish
Results
•Send
Notification/Email and
Monitor
Continuous Testing
Continuous Integration
DevOps - Automation
•Monitor Production
and Test environment
•Device Health
Monitoring
•Functional Alerts
•Automated
Troubleshooting
•Automated Email /
SMS Alerts
Continuous Test – Automation Solution
9
Continuous Test – Sensor Virtualization
• Telegesis Zigbee USB for
Sensor virtualization
• Sensors’ Firmware is
deployed on Telegesis USB
• Python Scripts triggers
command to Raspberry Pi to
generate specific signals
from virtualized Sensors
• Virtualized Sensors passes
signals to CPE over Zigbee
• CPE sends information to
Servers based on Signals
received from Sensors
• Python Scripts verifies actual
outcome from Servers
(Application logs as well as
Database
• XMPP Interceptor used for
testing of System behaviour
such as when there are
delays
• Load / Stress testing from
Touchscreen to Server
10
Continuous Test – Build Verification
11
Virtualization for Test Automation
• Raspberry Pi
• Web Power Switch
• Simulation of Cameras using Nvidia cards
• Simulation of Hardware Boards – QEMU
• Sensor virtualization with Telegesis USB
• Network Virtualization using XMPP Interceptor
• Serial Port communication with Device
• Local Data centre in Cloud (AWS/Azure)
12
Virtualization – Video Analytics
Object Tracking and
Analysis through
Video Surveillance
Automated
Test Scripts
Video Files Test Data
Verification of Alerts
by Database Query
Web &
Database
NVIDIA Card
Alerts
IOT Board with
Application / Algorithm
13
References & Appendix
• Computer Weekly, 2014, Time for DevOps to get out of the weeds.
http://www.computerweekly.com/opinion/Time-for-DevOpsto-get-out-of-theweeds
• www.tecadmin.net
14
Questions & Answers
15
Thank You!!!