[email protected]
+420 261 220 070
Comparison of the MindG wristband with the established MotionWatch
device for monitoring of physical activity and sleep
BACKGROUND
This document provides comparison of our physical activity monitoring device – the MindG wristband
– with the MotionWatch and corresponding analytical software: MindPax cloud and MotionWare.
MotionWatch is a monitoring device that has became a standard device for monitoring of physical
activity and sleep.
In our comparison, we focused on three main topics: (i) we compared the amplitude of the signals
measured by both devices, (ii) we evaluated the capabilities of the system in sleep detection, (iii)
we used the signals from both of the devices to assess the circadian parameters of the participants’
sleep. In order to perform the comparison, we organized a simultaneous measurement of 20 persons’
everyday activity with both devices for a period of two months.
METHODS
Amplitude Comparison
Sleep detection
The different approaches to raw accelerometric
data aggregation caused discrepancies in the
signals. Therefore, the data were filtered by
moving average filter of length equivalent
to one hour period to eliminate the effects
of aggregation. The signals from MindG and
MotionWatch were compared by the Spearman’s
correlation coefficient complemented by
95% confidence interval estimated by bootstrap.
The detection consists of two main
parts: (1) sample-based classification of the
actigrams into sleep and non-sleep and (2)
detection and grouping of the sleep intervals
(non-interrupted sleep) with subsequent
estimation of the fall-asleep and wake-up
times. The first part is done using a linear model
estimated using logistic regression; the model
is then used to obtain “sleepiness” measure of
each actigram sample. In the second part, the
sleepiness is thresholded to obtain the sleep
intervals that are then post-processed using
rule based algorithm to obtain the fall-asleep
and wake up times.
Office, R&D
MINDPAX s.r.o.
Oldřichova 49
128 00 Praha 2
“ We are helping people who suffer from bipolar affective disorder. ”
[email protected]
+420 261 220 070
Circadian rhythmicity analysis
800
600
400
200
MotionWatch
The nonparametric analysis of the circadian
rhythmicity was performed by computation
of the interdaily stability (IS), the interdaily variability (IV), the relative amplitude (RA), L5, L5
start time, M10 and M10 start time parameters
[1]. The computed values were summarised and
compared to a reference implementation [2]
by MotionWare software applying a resistant
regression analysis.
User 224 (r = 0.96)
0
RESULTS
Amplitude Comparison
0
The correlation analysis of the signals yielded
average correlation between the signals
0.738 (median 0.830), which indicates strong
relationship between the signals. Comparision
of amplitudes of both signals is depicted in Figure 1.
100
200
300
400
500
600
MindG
Figure 1: Example of amplitude comparision
Sleep detection
Three-fold cross-validation was used to assess the
performance of the algorithm. The performance
of the sleep model was measured by accuracy,
the Dice coefficient and the ROC curve resulting
in the accuracy 0.941, the Dice index 0.911 and
the area under ROC 0.969. The second part of
the detection - fall asleep and wake up times was assessed by the absolute deviance of the fall
asleep and wake up times from the ground truth
(sleep questionnaires) in minutes and resulted in
the mean error of 27 minutes for the fall asleep
time, 22 minutes for the wake up time. The results
are shown in Figure 2.
Office, R&D
MINDPAX s.r.o.
Oldřichova 49
128 00 Praha 2
Figure 2. Errors in the identification
of fall-asleep and wake-up times
“ We are helping people who suffer from bipolar affective disorder. ”
[email protected]
+420 261 220 070
Circadian rhythmicity analysis
The L5 start time value coincides among the
systems in 80.8% repeat of cases and the times
differing up to 2 hours constitute 89.1% of the
cases. The average difference in the L5 start
times between the MotionWatch and MindG is
7.64 mins. The M10 start time value coincides in
48.3% and 80.1% of the cases with the average
difference of 21.1 minutes.
User 213
Interdaily varability
1.0
1.1
0.40
1.2
Interdaily stability
Office, R&D
MINDPAX s.r.o.
Oldřichova 49
128 00 Praha 2
Nov 15
Dec 05
Nov 15
Nov 15
Dec 05
Dec 05
Least active 5 hour period
1000 2000 3000 4000 5000
0.70
0.80
0.90
Relative amplitude
Nov 15
Dec 05
25000
30000
35000
Most active 10 hour period
20000
The estimated robust model coefficients
show that the values of interdaily stability and
variability correlate satisfactorily between the
two systems - see Figure 3 - as indicated by
regression coefficients close to 1 (0.96 for IS, 1.13
for IV), with systematics average differences
(-0.09 for IS, 0.16 for IV). On the other hand,
the values are on average 0.11 lower in case of
interdaily stability and 0.16 higher in case of
interdaily variability. The remaining parameters
(RA, L5 and M10) show greater variability in the
relationship. The difference originates in the
different aggregation method as those values
are based on sum of the signal elements (the
L5 and M10 directly, the relative amplitude
as a function of L5 and M10). These values
are significantly lower in the MindPax cloud.
0.20
0.9
0.30
The nonparametric characteristics were computed in 7 days long sliding window. We compared the values from MindG system to those
computed on MotionWatch by linear regression
estimated by robust methodology. Firstly,
we estimated which data points are outliers
by robust least quantile regression (LQS) [3].
The data point whose residuals were higher
than 2.57 (0.005 quantile of standard normal
distribution) times residual variance estimate
from LQS model, were denoted as outliers and
excluded from further analysis. Secondly, the
resistant least trimmed squares regression (LTS)
was utilized to conclude the comparison [3].
-
Nov 15
In house implementation
MotionWare
Dec 05
Figure 3. Example of non-parametric parameters
comparison
“ We are helping people who suffer from bipolar affective disorder. ”
[email protected]
+420 261 220 070
CONCLUSION
The comparison showed the amplitude of signals is highly correlated, even though the method
of raw accelerometric data aggregation differs. The difference in sleep detection is sufficiently
accurate for use in scientific studies as well as for healthcare application. The comparison of circadian
parameters showed several systematic differences nevertheless for analysis of relative changes in
time or among groups the differences are negligible. Furthermore, in literature are available different
implementations of circadian parameters [5]. While MotionWare implementation origins from
classical paper of [1], the MindPax cloud implementation is based on newer contributions [4] which is
more suitable for clinical practice. More detailed information could be found in the Appendix which
accompanies this White paper.
REFERENCES
[1] VAN SOMEREN, Eus JW, et al. Bright light therapy: improved sensitivity to its effects on restactivity rhythms in Alzheimer patients by application of nonparametric methods. Chronobiology
international, 1999, 16.4: 505-518.
[2] Motionware software (version 1.1.12), CamNtech Ltd., Cambridge, United Kingdom, 2015.
[3] ROUSSEEUW, Peter J.; LEROY, Annick M. Robust regression and outlier detection. John Wiley & Sons,
2005.
[4] GONÇALVES, B. S. B., et al. A fresh look at the use of nonparametric analysis in actimetry. Sleep
medicine reviews, 2015, 20: 84-91.
[5] Kushida, Clete A., et al. Comparision of actigraphic, polysomnographic, and subjective assessment
of sleep parameters in sleep-disordered patients. Sleep medicine, 2001, 2.5: 389-396.
Office, R&D
MINDPAX s.r.o.
Oldřichova 49
128 00 Praha 2
“ We are helping people who suffer from bipolar affective disorder. ”