Journal of Motor Behavior, 2006, Vol. 38, No. 2, 139–159
Copyright © 2006 Heldref Publications
A Practical Guide to Time–Frequency
Analysis in the Study of Human
Motor Behavior: The Contribution
of Wavelet Transform
Johann Issartel
Ludovic Marin
Thomas Bardainne
University of Montpellier 1, Montpellier, France
University of “Pau et des pays de l’Adour,”
Pau, France
Philippe Gaillot
Marielle Cadopi
University of “Pau et des pays de l’Adour,” Pau, France
and Center for Deep Earth Exploration
Yokohama Kanagawa, Japan
University of Montpellier 1, Montpellier, France
ABSTRACT. The authors present a practical guide for studying
nonstationary data on human motor behavior in a time–frequency
representation. They explain the limits of classical methods founded exclusively on the time or frequency basis and then answer
those limits with the windowed Fourier transform and the wavelet
transform (WT) methods, both of which are founded on time–
frequency bases. The authors stress an interest in the WT method
because it permits access to the whole complexity of a signal (in
terms of time, frequency, amplitude, and phase). They then show
that the WT method is well suited for the analysis of the interaction between two signals, particularly in human movement studies.
Finally, to demonstrate its practical applications, the authors apply
the method to real data.
or the frequency basis, but never on both simultaneously.
Those methods cannot provide insight into the temporal
evolution of the frequency. Our aim in this tutorial is to present some methods that permit one to observe frequency
evolution with time in a nonstationary signal. The two main
methods that allow one to perform a time–frequency analysis are the windowed Fourier transform (WFT) and the
wavelet transform (WT). They allow one to depict a nonstationary signal in terms of time, frequency, amplitude,
and, eventually, phase. Those methods are used in lots of
experimental domains but not often in experimental psychology. We would like to help remedy that deficiency by
proposing a practical guide that lets one understand why
those methods are interesting and how they can be applied
in the domains in which we are interested.
We have organized this article in three different sections. First, we introduce the notion of temporal and frequency bases and the concept of time–frequency analysis
by proposing two approaches to time–frequency analysis,
namely, WFT and WT. In the second section of this article, we present the interaction between two people by
characterizing the nature of coordination in a human
behavioral motor task. We illustrate the ability of the windowed cross-correlation function (WCCF), and particularly the cross-wavelet transform (CWT), to enable us to
understand the interactions between two nonstationary
signals. In the first two sections, we use synthetic examples to help define the advantages and limitations of each
method. Finally, in the third section, we apply the WT and
Key words: cross-wavelet transform, motor behavior, time–
frequency basis, wavelet transform
I
n studies of human motor behavior, experimenters are
increasingly striving toward ecological situations. The
analysis of such situations is sometimes quite difficult
because one must take into consideration the evolution of
behavior over time. To that end, investigators need a specific method that enables them to analyze such changes over
time. For instance, consider the wrist movements of a tennis player over an entire set or match. The wrist may move
in different directions (right, left, forward, or backward),
quickly or slowly, and with high or low frequency and high
or low amplitude. The wrist’s trajectory becomes more
complex and obviously nonstationary as the frequency content of the signal evolves with time. It is quite difficult to
analyze the motor characteristics of such an individual complex signal, and it is even more difficult to study the interactions between two signals—in that particular case, the
movements of two tennis players’ wrists. In classical methods such as auto- or intercorrelation and the Fourier transform, one assumes the stationarity of the analyzed signal.
Above all, the methods are founded on either the time basis
Correspondence address: Ludovic Marin, UPRES EA 2991—
Motor Efficiency and Motor Deficiency Laboratory, University of
Montpellier 1, 700 avenue du Pic Saint Loup, 34090 Montpellier,
France. E-mail address: [email protected]
139
J. Issartel, L. Marin, P. Gaillot, T. Bardainne, & M. Cadopi
CWT methods to real data to illustrate their relevance in
human movement studies.
Time–Frequency Analyses
Notion of Temporal and Frequency Analyses—Local and
Global Concepts
To understand the time–frequency analysis of a signal,
we first present the limits of temporal and frequency analyses of a signal. To clarify our demonstration, we present all
analyses (time, frequency, and time–frequency) with a synthetic signal. That illustrative signal serves as a tool to help
us explain and compare different methods of analysis. The
synthetic time series has a duration of 204.8 s (4,096 data
points, 20-Hz sampling rate). One obtains synthetic signal
s1 (Figure 1D) by summing three sines from the following
general equation:
 2π t

s(t ) = A sin 
+ ϕ ,
 T

(1)
with a high (0.44 Hz, Figure 1A), an intermediate (0.16 Hz,
Figure 1B), and a low (0.08 Hz, Figure 1C) frequency, and
an amplitude A (= 150 arbitrary units). We have added local
nonstationary components such as amplitude modulation
(Aam) as obtained from the following equation,

 2π t  
 2π t

s(t ) = A 1 + Aam sin 
  sin  T + ϕ  ,
T



am 

(2)
(3)
and phase shifts, commonly encountered in movement studies, to those three main frequency components. The highfrequency component of s1 is amplitude-modulated in Time
Interval 3 (102.4–153.6 s). That amplitude modulation is
characterized by a period (Tam) of 78.8 s (corresponding frequency ~0.013 Hz) and an amplitude (Aam) equal to 3. The
intermediate-frequency component of s1 is frequency modulated in Time Interval 2 (51.2–102.4 s). That local frequency
modulation is defined by a period (Tfm) equal to 50 s (corresponding frequency = 0.02 Hz) and an amplitude (Afm) equal
to 1.5. Restricted to Time Interval 4 (153.6–204.8 s), the
phase shifts of the high- and low-frequency components are
equal to +30° and –135°, respectively. All of the justdescribed properties, when taken together, yield a nonstationary signal. Details of the various components of s1 are
listed and displayed in Figure 1.
In classical experiments, investigators have covered various temporal scales, ranging from few seconds, as in a tapping task (Gilden, Thornton, & Mallon, 1995), to hours, as
in a learning process (Zanone & Kelso, 1992). Those experiments were recorded on a temporal basis. The well-known
140
∞
S (ω ) =
frequency modulation (Afm), as obtained from the following
equation,
  2π t 

 2πt 
s(t ) = A sin 
+ Afm 
+ ϕ ,


 Tfm 
 T 

temporal basis is the one we deal with every day, and it is
defined by a series of unitary local impulse functions, called
Dirac, d(t), with a time (t) gap equal to the chosen sampling
rate (Chatfield, 1989). The first step in the analysis of any
signal lies, simply and naturally, in its visual inspection.
One can also use complementary quantitative methods
based on the analysis of the temporal variation of various
statistical parameters, such as computation of the mean and
the variance with a mobile window. The use of different
windows allows for the extraction of global information
averaging the different frequency components of the analyzed signal over different durations. The results obtained
with that kind of analysis, however, depend on the chosen
window duration. Thus, the interpretation of such complementary temporal information is difficult, indeed even
impossible for nonstationary signals.
In addition to the temporal analysis, the most common and
widely used transformation is the Fourier transform (FT). In
the FT, the time series is transformed from the standard time
domain to the frequency domain. The FT consists of a decomposition of the analyzed signal s(t) into a combination of harmonic functions (also called an analyzing function), that is,
sines and cosines that together result in a global description of
the frequency content of the analyzed signal, S(ω),
∫ s(t )e
− jωt
dt.
(4)
−∞
To obtain a global description of the frequency content of
the analyzed signal, one commonly uses spectral (or Fourier) analysis. The Fourier power spectrum of s1 is presented
in Figure 2. The Fourier power spectrum of s1 displays
three main peaks around 0.44, 0.16, and 0.08 Hz associated
with the high-, intermediate-, and low-frequency components of s1, respectively. Because there is no temporal information, the other peaks with lower amplitude between
0.14–0.18 Hz and 0.43–0.46 Hz are difficult to interpret.
For nonstationary signals, interpretation of temporal
analysis results without any frequency component is complex, as is interpretation of frequency analysis results in the
absence of any temporal information. Joint analysis of independent temporal and frequency results does not provide
any complementary information. Indeed, averaging information in the time domain independently from that in the
frequency domain (temporal analysis) or, conversely, analyzing frequency domain information independently from
temporal information (frequency analysis) provides global
information. Those two extreme representations of the same
signal cannot be combined in a common framework in
which frequency information would be localized in time.
Local information is thus crucial for the analysis of nonstationary signals and argues for the use of the time–frequency
analysis method. To stay in a practical spirit, let us consider a common example that highlights the importance of the
information on the time–frequency plane: reading a piece of
music (Figure 3).
Journal of Motor Behavior
Time–Frequency Anaylsis of Human Motor Behavior
Interval
2
3
[51.2–102.4]
[102.4–153.6]
1
[0–51.2]
A
4
Frequency
[153.6–204.8] Component
100
High
f ≈ 0.44 Hz
A = 150
ϕ = 0º
0
–100
(1)
Aam = 3
Tam = 78.8s
Phase-shift
ϕ = 30º
(1)
B
100
Intermediate
f ≈ 0.16 Hz
A = 150
ϕ = 0º
0
–100
(2)
Afm = 1.5
Tfm = 50s
(2)
C
100
Low
f ≈ 0.08 Hz
A = 150
ϕ = 0º
0
–100
D
Phase-shift
ϕ = 135º
400
Signal
s1
0
–400
0
50
100
Time (s)
150
200
FIGURE 1. Representation of signal s1 used as an example in this study. One sums three different sines (A, B, and C) to obtain s1 (D). The high-frequency component (A) is modulated
in amplitude in Time Interval 3 (102.4–153.6 s) and is phase shifted 30° in Time Interval 4
(153.6–204.8 s). The intermediate-frequency component (B) is modulated in frequency in
Time Interval 2 (51.2–102.4 s). The low-frequency component (C) is phase shifted 135° in
Time Interval 4 (153.6–204.8 s; see general equations). A = amplitude; f = frequency; ϕ =
phase; Aam = amplitude modulation; Tam = period of the amplitude modulation; Afm = amplitude of the frequency modulation; Tfm = period modulation.
A piece of melody (a time series) may be written as a
musical score (a time–frequency representation of the time
series). Each note represents a musical tone (a part of the
signal) that is characterized by four parameters: (a) frequency (indicated by vertical location), (b) time of occurrence (indicated by a horizontal location), (c) time duration
(indicated by the tempo and by different types of notes), and
(d) intensity (indicated by fs and ps as well as by accents
and crescendo and decrescendo symbols). The musical
notes represent different time lengths on a dyadic scale, for
example whole note, half note, quarter note, and eighth
note. It is obvious that if all the local information is omitted, then there is little left in a piece of music. For example,
March 2006, Vol. 38, No. 2
if we count the occurrence of each tone in a piece of music,
scaled by the same note unit, then we find that the dominant
tones are the elements of the main chord that belong to
either a major or a minor scale. The scale, for example, C
major or A minor, is the global information of a melody that
is usually given at the beginning of the musical score. That
global information (equivalent to a Fourier transform) says
very little about the real content of the music, however, and
many different kinds of music may share the same global
information (written, say, in C major). In fact, that global
information corresponds to the dominant peaks in the power
spectrum of a time series. Thus, we can see a strong parallel here between music and motor time series analysis. As is
141
Fourier Power Spectrum
0.1
0.2
0.44 Hz
100
0.16 Hz
150
0.08 Hz
J. Issartel, L. Marin, P. Gaillot, T. Bardainne, & M. Cadopi
75
50
25
0
0.3
0.4
0.5
0.6
Frequency (Hz)
Frequency (Hz )
FIGURE 2. Frequency (Fourier) analysis of the synthetic
signal s1. The Fourier power spectrum clearly displays
three peaks at frequencies 0.08, 0.16, and 0.44 Hz. Other
minor peaks or fluctuations around the three main peaks are
often ignored because they are very difficult to interpret
without any temporal information.
698.5
587.3
493.9
392.0
329.6
WFT (Gabor Transform)
0
1
2
Time (s)
FIGURE 3. The musical score, a good example of
time–frequency representation. Such a representation
allows one to simultaneous represent the location in time,
frequency, duration, and intensity of a given event (note).
the case with music, to fully define a motor behavior time
series, one needs to preserve both local and global information. As we attempt to show in the following pages, like the
musical score, the time–frequency analysis is the simplest
and most natural way to quantify the frequency evolution of
a signal with time.
The time–frequency plane is defined by the time interval
that spans the signal (or part of it, if necessary) and by frequency, which ranges from zero to the Nyquist frequency,
that is, 1/2 δ(t), where δ(t) is the sampling interval. The idea
of a time–frequency analysis is to “tile” a time–frequency
plane (musical score) with rectangular tiles, usually called
Heisenberg cells, and assign to each cell a magnitude representing the power of the signal in the time–frequency
142
interval spanned by the cell. As we discuss later, the minimum area of those cells is determined by the uncertainty
principle, which dictates that one cannot measure with arbitrarily high resolution in both time and frequency. The way
the tiling of the plane is done depends on the basis one
chooses to represent the signal (Figure 4). When we use the
standard basis in the time domain, that is, Dirac functions,
we can localize the process very well in the time domain but
not at all in frequency domain. Tall thin cells in Figure 4A
schematically depict that property. In the case of Fourier
bases, we get exact localization in frequency but none in
time, as is depicted by the long horizontal cells in Figure
4B. In those cases, as in the following ones, the decomposition pattern of the time–frequency plane is predetermined
by the choice of the analyzing function.
Although there currently are a few different approaches
to time–frequency analysis (Feichtinger & Gröchenig,
1988; Kumar & Foufoula-Georgiou, 1997), we focus in this
guide exclusively on those that are specifically pertinent to
our domain: The well-known WFT and the WT (Chui,
1992a; Daubechies, 1992; Torrence & Compo, 1998). In
considering the other possible approaches that we do not
discuss in this article, one can refer to the now-outdated
Wigner-Ville distribution (Cohen, 1989) and to approaches
based on adaptive kernels (Baraniuk & Jones, 1993; Jones
& Baraniuk, 1995; Mann & Haykin, 1995). Note that adaptive kernel methods are much more efficient than the WT in
the analysis of single time series. One cannot use those
methods simply to study the interactions (e.g., synchronization, coordination) between signals that result from local
adaptation of the kernel to the signal, however, and the
methods are thus beyond the scope of this article.
Named for D. Gabor following his fundamental work
(Gabor, 1946), the Gabor transform (GT) includes and can
be illustrated by a technique known as the windowed Fourier transform (also called short-time Fourier transform). We
show later on that most parts of the theoretical concepts of
WFT will be used in the WT. One performs the WFT by
first dividing the analyzed signal into short consecutive
(overlapping or not) segments and then computing the
Fourier coefficients of each segment (the same analyzing
function as FT). The windowing of WFT allows for the
introduction of temporal information. To reduce “leakage”
aberrations in the Fourier output that are introduced by sudden changes at the start and end of data, one usually
smoothes segments by using a windowing function g that
smoothly decreases to zero at each end of the data. Let s(t)
be the signal and g(t) an ideal cutoff window. Chopping up
the signal amounts to multiplying s by a translate of g, that
is, by, g(t – s) (because g is real, the conjugate is irrelevant
here). The local Fourier coefficients of that product are then
∞
S (ω , τ) =
∫ s(t )  g(t − τ)e
−∞
− jωt
 dt.

(5)
Journal of Motor Behavior
Time–Frequency Anaylsis of Human Motor Behavior
Basis Function
Time-Frequency Plane
Standard
(time)
Frequency
A
Time
0
Time
Fourier
(frequency)
Frequency
B
Time
Time
Windowed Fourier
(time-frequency)
Frequency
C
0
0
Time
Wavelet
(time-frequency)
Frequency
D
Time
Time
0
Time
FIGURE 4. Temporal basis (left) and tiling of the time–frequency plane (right) for (A) standard (time), (B) Fourier (frequency, FT), (C) windowed Fourier transform, and (D) wavelet
transform.
March 2006, Vol. 38, No. 2
143
J. Issartel, L. Marin, P. Gaillot, T. Bardainne, & M. Cadopi
Because that product is classically used with the FT, the computation of the WFT permits us to extract information about
amplitude and phase. The information given by the phase is
rarely used on a single signal because the result of averaging
operations over the time window interval is often hard to
interpret. However, the notion of phase could be interesting in
the study of the interaction between signals with the method
of the cross-WFT. Let us now look at the behavior of the window function in both the temporal and the frequency domains.
The temporal axis is subdivided into widths equal to the duration of the window interval, and the frequency axis is subdivided into widths corresponding to the frequency resolution
of the analysis (Figure 4C). The width in the temporal domain
is directly coupled with the width in the frequency domain
according to Gabor’s (1946) analogon of the Heisenberg
uncertainty criterion. The sharpness of the time analysis can
be traded off for sharpness in frequency, and vice versa.
Before presenting the results given by the WFT on s1, we
first explain how to read a WFT representation. One can
obtain a reading of the WFT spectrum, also called a spectrogram and defined as the squared magnitude of the WFT (see,
e.g., Figure 5), by constructing a gray-scale diagram in which
colored patches represent the spectrum of the WFT; time is on
the horizontal axis, and frequency is on the vertical axis.
Whereas the time axis is obviously presented in a linear scale,
linear or logarithmic scales can be used for the frequency axis.
For long-term signals, signals containing scales that differ by
many orders of magnitude (many decades), or both, a linear
scale should not be used because it is imprecise for long periods and requires extensive computations. In such a case, a
log10-based scale or an octave-band decomposition (log8based scale), as in music, is preferable. A good compromise
between the linear scale (e.g., right vertical axis in Figure 5)
and the log10-based scale is the log2-based scale (also called a
dyadic scale). In the rest of this article, we use the dyadic
scale and show its natural legitimacy when one uses the WT.
The left vertical axis in Figure 5 represents the frequency in
the dyadic scale. In the reading of the spectrogram, the dashed
areas are other important elements. Those areas outline edge
effects that have an extension proportional to the window
width (duration). We use those kinds of dashed areas throughout the article to define the edge effects.
To facilitate the interpretation of that type of diagram, we
can define a level (black lines) above which a maximum in the
local WFT spectrum is statistically significant. That test was
widely inspired by a test detailed for the wavelet spectrum by
Torrence and Compo (1998). The main idea behind those tests
is that each point of the WFT or the WT power spectrum is
statistically distributed as a chi-square (χ2) with 1 degree of
freedom (df) for real analyzing functions (in the case of WFT
or a real wavelet mother function, see the WT section) and 2
df for a complex analyzing window (e.g., the wavelet Morlet
mother function used in this article; see the WT section). The
confidence level at each frequency is therefore the product of
the background spectrum and the desired significance level
(e.g., 95% confidence) from the χ2 distribution. Situations in
144
which the background spectrum is known are so rare that one
practically determines the background spectrum by calculating the time-average of the spectrum (Torrence & Compo). As
we discuss later, the threshold level and resulting contours on
spectra of those tests, just like any other significance test,
must be interpreted carefully. Looking at how large the 95%
regions are and how they are organized can help in determining whether that is a significant result (Torrence & Compo).
The next step is to extract some variables and to perform classic quantitative comparisons between experimental conditions. We cite a few examples in the section on the WT.
WFT analyses of s1 generated from windows of duration
12.8 s (256 points), 25.6 s (512 points), and 51.2 s (1,024
points) are presented in Figure 5. The calculation with the
window of duration 25.6 s (Figure 5C), in Time Interval 1
(0.0–51.2), showed that the stationary components (horizontal significant zones) were clearly identifiable. The amplitude
modulation of the intermediate-frequency band in Time Interval 2 (51.2–102.4 s) is well outlined by an oscillating arch
ranging from ~ 0.13 to 0.17 Hz. Because of the decrease in
amplitude of the high-frequency component in Time Interval
3 (102.4–153.6 s), the amplitude modulation appeared weak
and was not judged statistically significant. The statistical test
mentioned earlier must be interpreted with caution. It simply
informs us that the energy of the high-frequency band in that
portion of the signal is weak with respect to that of the other
components of the analyzed signal. On the contrary, another
significant oscillating arch, located at the limit between Time
Intervals 3 and 4 (t = 153.6 s), reflected a local frequency
change associated with the major phase shift (–135°) of the
low-frequency band in Time Interval 4 (153.6–204.8 s). The
minor phase shift (+30°) of the high-frequency band in the
same time interval was not evidenced because of the weak
amplitude of that component at t = 153.6 s. Reducing the window duration to 12.8 s (Figure 5B) allowed us to better define
the amplitude modulation of the high-frequency component
in Time Interval 3. On the other hand, the frequency resolution was decreased (thicker significant zones), resulting in
interactions between the intermediate- and low-frequency
components in Time Interval 2 that limited the characterization of the frequencies, and particularly the low frequencies.
On the contrary, increasing the window duration to 51.2 s
(Figure 5D) led to a very accurate frequency resolution but
restricted the temporal domain of investigation by 25% (i.e.,
25.6–179.2 s). Another drawback to the use of long-duration
windows is that short-term nonstationarities (much shorter
than the analyzing window) have been detected with a poor
temporal accuracy. Figure 5E represents the phase of the WFT
with the window duration of 25.6 s. To ensure the figure
would not be overloaded, we have not presented the other figures of the phase (window durations 12.8 s and 51.2 s). We
have kept only the figure with the window duration of 25.6 s
(Figure 5E) because that figure possesses the better time–
frequency tradeoff and consequently the best results. Figure
5E should give some information about the time of the phase
changes that appeared in the signal (+30° modification of the
Journal of Motor Behavior
Time–Frequency Anaylsis of Human Motor Behavior
A
400
200
0
–200
–400
Signal
s1
50
100
150
200
–6.00
–6.75
0.16
–7.50
–8.25
–6.00
–6.75
0.16
–7.50
51.2 s Window
0.08
Frequency
(Hz)
D
0.44
–8.25
–5.25
–6.00
–6.75
0.16
–7.50
0.08
–8.25
0
50
100
Time (s)
Spectrum
Min.
Frequency
(Hz)
E
0.44
25.6 s Window
Frequency
(2nHz)
–5.25
Frequency
(2nHz)
25.6 s Window
0.08
Frequency
(Hz)
C
0.44
Frequency
(2nHz)
–5.25
150
200
Max.
–5.25
–6.00
–6.75
0.16
–7.50
0.08
Frequency
(2nHz)
12.8 s Window
Frequency 0
(Hz)
B
0.44
Amplitude
Interval
2
3
4
[51.2–102.4] [102.4–153.6] [153.6–204.8]
1
[0–51.2]
–8.25
0
50
0
100
Time (s)
Phase (º)
150
200
360
FIGURE 5. Windowed Fourier analysis of synthetic signal s1. (A) Representation of signal
s1. Windowed Fourier power spectrum obtained for (B) 12.8-s, (C) 25.6-s, and (D) 51.2-s
cosine windows. Frequencies are presented on a dyadic scale (power of 2 scale). Note in C
(a) the oscillating arch typical of frequency modulation for the intermediate frequency band
in Time Interval 2 and (b) the bridge associated with the phase shift of the low-frequency
component in between Intervals 3 and 4. Selecting a short duration window (B) provides a
better temporal resolution—the amplitude modulation of the high-frequency band in Time
Interval 3 is more visible, but prevents analysis of long term fluctuations. Increasing the duration of the window (D) reduces the temporal resolution of the results and increases edge
effects (shaded area). Note that, independently of the window duration, the frequency resolution depended on the analyzing frequency. (E) Phase of the windowed Fourier transform of
s1 on the 25.6-s window duration.
March 2006, Vol. 38, No. 2
145
J. Issartel, L. Marin, P. Gaillot, T. Bardainne, & M. Cadopi
phase of the high-frequency component and –135° modification of the low-frequency component at the beginning of
Interval 4). As we explained earlier, the current figure cannot
give us those explanations because of the limitations of such
an analysis.
One problem with such a time–frequency localization
technique is that one computes the frequencies associated
with only a small portion of the signal. Consequently, it
poorly resolves phenomena with durations shorter than the
time window. Moreover, shortening the window to increase
time resolution can result in unacceptable increases in computational effort, especially if the short-duration phenomena
being investigated do not occur very often. Taking a small
portion of the signal provides a good localization in time but
prevents investigation of low-frequency (long-term) components. Therefore, although the WFT gives us a regular tiling,
whatever the frequency of analysis, the tradeoff is not good
because of the imprecision in both time and frequency (Figure 4). One would have to have a short window for the high
frequency and a long window for the low frequencies. As we
discuss next, the WT, based on another set of analyzing
functions, does not present those limitations.
WT
Since its introduction in seismics by Morlet (1983) over
two decades ago, WT has found wide application in diverse
fields of geosciences (see Foufoula-Georgiou & Kumar, 1994,
for a review). The WT method is also now being used across
a number of domains in physiology (Jobert, Tismer, Poiseau,
& Schulz, 1994), particularly in electroencephalographic (De
Carli et al., 2004; Sakowitz, Quiroga, Schurmann, & Basar,
2001) and electromyographic (De Michele, Sello, Carboncini,
Rossi, & Strambi, 2003) analyses, as well as in the study of
heart disease precursors (Li, Zheng, & Tai, 1995) and in
human visual channels (Gaudart, Crebassa, & Petrakian,
1993). Most recently, WT has been used in communication
(Visser, Otsuka, & Lee, 2003) and neuroscience (Armand &
Minor, 2001) studies. The WT is an analysis tool well suited
to investigating nonstationary signals that characterize complex human motor behavior that occurs over finite temporal
domains. For example, the WT analysis would permit us to
study the movement of a tennis player’s wrist throughout an
entire match. It seems somewhat surprising, then, that there is
still a dearth in the use of WT in analyses of motor behavior.
One our motivations for this article was to remedy that situation and to call attention to the usefulness of WT as a powerful tool for experimental psychology research that can provide
a new way to focus our attention on human motor behavior.
The WT offers a better tiling of the time–frequency plane
than the WFT does (Figure 4D). Indeed, based on an octave
band, or, more precisely, multiresolution decomposition (Figure 4D), one can well localize the highest frequencies in time
and accurately quantify the lowest frequency components.
Obviously, according to the Gabor (1946) analogon, the
uncertainty in frequency localization increases as the uncertainty in time decreases. That change in uncertainty is reflect146
ed as taller, thinner cells with increase in frequency. Consequently, the frequency axis is finely partitioned only near low
frequencies (Figure 4D). One obtains such a tiling of the
time–frequency plane naturally by dilatation and contraction
of an analyzing window, called the mother function. The WT
of a signal s(t) is defined as the integral transform
S (λ , τ) =
∫
∞
−∞
s(t )φλ ,τ (t )dt ,
(6)
where φλτ = (1/λ1/2)φ([t – τ]/λ) represents a family of functions called a daughter function or an analyzing function
because one derives it from the mother function φ by dilatation or contraction (λ) and translation (τ) wavelets
(Daubechies, Grossmann, & Meyer, 1986). Here, λ is a
scale (dilatation or contraction) parameter that can be linearly converted to frequency ω, τ is a location parameter
(translation), and φλ,τ(t) is the complex conjugate of φλ,τ(t).
Therefore, the WT provides a flexible time scale window
that narrows when focusing on small-scale features and
widens on large-scale features, analogous to a zoom lens on
a camera. The WT may be seen as a mathematical microscope, where the magnification is given by 1/λ and the
optics are given by the choice of the wavelet φλ,τ. It is
important to note that φλ,τ has the same shape for all values
of λ (Figure 4D).
The choice of the mother function φλ,τ is neither unique
nor arbitrary. It is a function with unit energy, chosen so
that it has (a) compact support, or sufficiently fast decay,
which enables one to obtain localization in time, and (b)
zero mean. The requirement of zero mean is called the
admissibility condition of the wavelet. The mother function φλ,τ is called a wavelet because of the just-noted properties. The second property ensures that the mother function φλ,τ has a wiggle (i.e., is wave-like), and the first
ensures that the mother function is not a sustaining wave.
As is evident, the two conditions just described leave open
the possibility of using several different functions as
wavelets (Torrence & Compo, 1998). The Mexican Hat
and the Morlet wavelets are popular (see Daubechies,
1992). The Mexican Hat (Figure 6A) is a real wavelet. The
Morlet wavelet (Figure 6B) has a complex value that
enables one to extract information about the amplitude
and phase of the process being analyzed. That wavelet is
defined by an order that controls the number of periods of
the mother function. As we discuss later, the use of a complex wavelet with phase computation is fundamental in
quantifying the interaction between signals. We used that
mother function for the analysis of s1 because it is polyvalent in analyzing nonstationary signals.
The WT is an invertible transformation (note that the WFT
is not invertible). In other words, independent of the choice
of the mother function, the obtained representation contains
the same amount of information as the original time series
(minimum condition for inversion and reconstruction).
Flandrin (1988) proposed calling the representation of
the WT a scalogram. Following the nomenclature introduced previously for the WFT, in the present article we refer
Journal of Motor Behavior
Time–Frequency Anaylsis of Human Motor Behavior
1.0
Normalized Amplitude
A
0.0
Mexican Hat
–1.0
–1
0
1
Time (s)
1.0
Normalized Amplitude
B
0.0
Morlet
–1.0
–1
0
1
Time (s)
FIGURE 6. Temporal representation of (A) the Gaussian
(Mexican Hat) mother function and (B) the Morlet mother
function. The Morlet mother function is a complex-valued
function (real part = solid line, imaginary part = dashed
line) allowing a phase measure. During wavelet computation, one obtains the analyzing wavelets (analysis function)
by dilatation and contraction and translation of the mother
function.
herein to the squared magnitude of the WT as the WT spectrum. Like the WFT spectrum, the WT spectrum (or its
square root, i.e., modulus) provides an unfolding of the
characteristics of a process on the time scale (time–
frequency) plane. Using the complex Morlet mother function (Order 8), we show wavelet analysis of s1 on the same
dyadic scale as in the WFT analysis of s1 in terms of spectrum and phase (Figure 7). As in the WFT (Figure 5),
dashed areas delimit edge effects. Because of different
dilatations and, thus, support of the analyzing wavelet for
March 2006, Vol. 38, No. 2
the different frequencies, edge effects are naturally coneshaped and not rectangular as in WFT.
Moreover, as shown in the WFT, thin black lines outline
statistically significant zones that are composed of a set of
coefficients. That set of coefficients is a matrix of numbers
that one can extract for further quantitative analysis. For
example, from such quantification one can obtain one or the
whole band of frequencies, which can be used to calculate
classical variables, for example, standard deviation and dispersion. It is also possible to extract different bands of time so
that one can compare the difference between the frequencies
for a given time. Thus, that method permits accurate access to
the local information. When variables are extracted, one can
perform comparisons with statistical methods that are classically used, such as the Quade test, the independent-sample
Mann–Whitney U, examination of slope regressions, and
multivariate analysis of variance (Karrasch, Laine, Rapinojac,
& Krause, 2004; Pajares & De la Cruz, 2004). In an effort to
focus on the contribution of the WT in our scientific domain,
we do not detail the applications of such statistical methods on
a set of variables in this tutorial.
The main difference from the WFT is that the presence of
multiscale structures and their temporal locations is more
easily and more precisely identified. The spectrum (Figure
7B) exhibits the (stationary or not) characteristics of the
three frequency components. Even if the spectrum is still not
statistically significant, the amplitude resolution is better
resolved than in the WFT, because wavelet support (duration) at the corresponding frequency is shorter than the 12.8-s
window used in the WFT, proving the advantage of scaledependent tiling over constant rectangular tiling. The phase
diagram (Figure 7C) is not commonly used in the literature
about wavelet analysis but can be very useful in localizing
singularities and nonstationarity present in the signal. In Figure 7C, the lines that shape “cones” are called isophase lines
and have the property of converging in situations in which
singularities occur, such as temporal changes, frequency
changes, or phase shifts. To understand those lines in a general manner, one needs to locate their start at the frequency
at which there is a modulation (as indicated in areas where
the thick lines are no longer parallel) and follow them until
their point of convergence at the top of the scalogram. The
localization of the start permits us to determine the frequency at which modification of the signal is produced. The point
of convergence enables us to localize the time when the singularities occur. As illustrated in Figure 7C, isophase lines
converge where a major modification has been introduced
(frequency modulation, amplitude modulation, and abrupt
phase shift). One can then use the accurate timing of those
events to further investigate the processes responsible for
such observations. For example, in Figure 7C, at the beginning of Time Interval 4 (153.6–204.8 s), we applied a phase
change of +30° on the high frequency and –135° on the low
frequency. We can clearly observe the isophase lines, which
converge toward the 153.6th second (the white circle at the
beginning of Interval 4). That method enabled us to accu147
J. Issartel, L. Marin, P. Gaillot, T. Bardainne, & M. Cadopi
Interval
2
[51.2–102.4]
3
4
[102.4–153.6] [153.6–204.8]
A
400
200
0
–200
Amplitude
1
[0–51.2]
–400
0
50
100
150
200
B
0.16
Frequency
(Hz)
0.44
0.08
C
0.16
Frequency
(Hz)
0.44
0.08
0
Min.
50
Spectrum
100
Time (s)
Max.
0
150
Phase (º)
200
360
FIGURE 7. Wavelet analysis of the synthetic signal s1 using the Morlet mother function. Representation of (A) signal s1, (B) wavelet transform spectrum, and (C) phase presented in the
same dyadic scale as in Figure 5. Because of the time and frequency properties of the wavelet
transform, all features, that is, (a) the amplitude modulation of the high-frequency component
of Time Interval 3, (b) the frequency modulation of the intermediate frequency component in
Time Interval 2, and (c) the phase shift that produced a local higher frequency component in
the low-frequency component in between Time Intervals 3 and 4, were detected with the same
temporal and frequency resolution for all the frequency components. In (C), the cone-shaped
lines represent areas where the singularities appear, such as the phase shift (white circle at the
beginning of Interval 4). Note that edge effects are cone-shaped because they depend on the
width of the analyzing wavelet, which itself depends on the analyzing frequency.
rately identify what frequencies were concerned by the
phase changes. We later used those properties to quantify the
difference of phase between signals.
To stay in the spirit of this tutorial, we will illustrate two
frequently asked questions from a novice in WT. The
answer to those questions provides complementary information and permits the reader to grow accustomed to this
new method. The questions frequently asked are about (a)
the choice of the mother function and (b) the order and the
kind of signals that can be analyzed with the WT.
The Mother Function and the Order
In the example just presented (Figure 7), we used a mother
function called Morlet. We chose the order 8 for the calculation because it is most suitable for our data (Signal 1). In this
148
paragraph, we show two extreme orders, 4 and 16, to illustrate
the kind of results that could be obtained in those cases. As we
explained earlier, the choice of the mother function is traditionally exposed and justified by the likeness between the signal and the mother function. It is necessary, however, to moderate those explanations because the order of the mother
function has a major influence on the frequency and the temporal resolution. The tiling of the time–frequency plane is
dependent on the choice of order. Thus, because of that property, the choice of the mother function on the final results
becomes minor.
Figure 8 represents the WT of Signal 1 with the two
extreme orders of Morlet. In Figure 8A, there is a Morlet
with an order of 4; in Figure 8D, a Morlet with an order of
16. Figure 8B shows that an order of 4 produces a good
Journal of Motor Behavior
March 2006, Vol. 38, No. 2
0
0
Min
50
50
100
Time (s)
Spectrum
150
150
Max
200
200
4
3
[102.4–153.6] [153.6–204.8]
100
2
[51.2–102.4]
0.08
0.16
0.44
0.08
0.16
0.44
Frequency (Hz)
0
0
0
1
[0–51.2]
50
50
D
150
150
360
200
200
4
3
[102.4–153.6] [153.6–204.8]
100
Time (s)
Phase (º)
100
2
[51.2–102.4]
F
E
FIGURE 8. Wavelet analysis of the synthetic signal s1, using the Morlet mother function with two extreme orders: an order of 4 (A) and an order of 16 (D). B and C represent the
WT spectrum and phase of s1, respectively, from the mother function with an order of 4. E and F represent the wavelet transform spectrum and phase of s1, respectively, from the
mother function with an order of 16. See also Figure 7 caption.
C
B
1
[0–51.2]
A
Time–Frequency Anaylsis of Human Motor Behavior
149
J. Issartel, L. Marin, P. Gaillot, T. Bardainne, & M. Cadopi
localization in time. Frequency modulation as well as localization in time are precisely detected, as can be seen in
Interval 2 (51.2–102.4 s) on the intermediate frequency. We
can precisely detect the start of the amplitude modulation in
Interval 3 (102.4–153.6 s) on the high frequency. On the
other hand, order 4, contrary to our example in Figure 7
(order 8), induced less precision in frequency and a disappearance of some statistical effects. For an order of 8 (Figure 7C), the isophase lines converge to the location of the
singularities, but, because of the order of 4, it is difficult to
observe them (Figure 8C). The augmentation of the order to
an extreme value such as an order of 16 produces an accurate localization of the frequencies and, consequently, a
diminution of the temporal resolution (Figure 8E). Thus, the
frequency modulation of Interval 2 (51.2–102.4 s) is not
clearly observable, and, overall, that modification of the
temporal resolution influences the statistical properties. The
significant areas become extremely fine, and the time duration of those areas decreases. We can observe in Figure 8F
that the isophase lines are particularly identifiable at the
beginning or the end of each interval. When the order of the
mother function is high, it then covers a more important
portion of the signal. We can observe the discontinuities for
a long time. To summarize, whatever the mother function,
the quantity of information is the same before or after the
transformation. Thus, the properties of the signal are always
expressed as a function of the time–frequency compromise
with the same information. The order of the mother function affects, above all, the tiling of the time–frequency
plane. In conclusion, the Morlet mother function with an
order of 8 could be a good starting point from which to analyze traditional signals of motor behavior.
Kinds of Analyzable Signals
Because of its intrinsically multiscale and local properties, the WT is particularly useful for the detection of small
structures superimposed on the gradient of larger ones in
describing individual time series. A constant signal produces
null coefficients (equal-admissibility condition). In contrast,
if the signal presents irregularities, then the wavelet reacts by
producing nonnull coefficients. Thus, all kinds of nonconstant signals can be analyzed with the WT, such as, for
Interval
2
3
4
[51.2–102.4] [102.4–153.6] [153.6–204.8]
1
[0–51.2]
200
0
–200
–400
0
50
100
150
Amplitude
400
A
200
B
0.16
0.08
C
Frequency
(Hz)
0.44
0.16
0.08
0
50
Spectrum
Min.
100
Time (s)
Max.
0
150
Frequency
(Hz)
0.44
200
Phase (º)
360
FIGURE 9. Wavelet analysis of the synthetic signal s1, using the Morlet mother function
with an addition of white noise with an amplitude of 400. The values of the white noise were
randomly distributed between 0 and 400 at each time sample. (A) Signal s1; (B and C) the
wavelet transform spectrum and phase of s1, respectively. See also Figure 7 caption.
150
Journal of Motor Behavior
Time–Frequency Anaylsis of Human Motor Behavior
example, continuous drift in the mean, a sudden change in
the mean, brutal change of frequency, and a noisy signal. We
chose to add a white noise with amplitude of 400 to the signal. The values of the white noise were randomly distributed
between 0 and 400 at each sample of time. We chose an
extreme case (such a noise is rarely present in motor behavior) to show that the WT could still discriminate the range of
frequencies with an important noise. The WT has the property of making localization in band-pass filters. The method
is insensitive to frequency contributions that do not overlap
with their frequency band. Hence, whatever the amplitude of
the white (high-frequency) noise, here chosen to equal 400,
results are not impaired by the addition of noise (Figure 9).
In Figure 9, we can observe the essential modulation contained in the signal as in Figure 7. On the intermediate frequency of the second interval, we can observe the modulation of frequency, and in the third interval, the amplitude
modulation. In this example, we can observe that the WT
method gives us coherent results with such extreme noise.
For further synthetic examples and applications of the WT
on nonstationary signals, the reader can consult Chui
(1992a, 1992b); for specific examples on colored noises, see
Bullmore et al. (2001) and Carevic (2005).
To generalize, the WT octave-band decomposition provides
an accurate time localization of short-time (high-frequency)
events and accurate frequency localization of long-term (lowfrequency) components, an advantage over WFT. Having
selected a given mother function (i.e., basis), wavelet decomposition offers a common basis for investigating interactions
between time series in terms of time, frequency, intensity, and
difference of phase. The interaction between two signals is
called the cross-wavelet transform (CWT). That interaction is
the scope of the following section.
stationary and contains the same three main frequency components of s1 (0.44 Hz, 0.16 Hz, and 0.08 Hz), with the
same amplitude (A = 150) and the same phase (ϕ = 0).
Interactions Between Signals: Cross-Analysis
where µ(Wx) and µ(Wy) express the means and SD(Wx) and
SD(Wy) express the standard deviations of the windows Wx
and Wy, respectively.
An assumption of stationary data is formulated within the
window. Thus, by moving the window gradually, step by
step, we have an estimation of the time series relationship.
The window permits us to observe the dynamic evolution of
the association between the two time series. To begin the calculation, one must determine four input parameters: (a) window size, (b) window increment, (c) maximum lag, and (d)
lag increment. Boker et al. (2002) explained that one should
choose those parameters as small or as long as possible, but
large enough or small enough so that one can observe a
change in time series. The detailed procedures are presented
in Boker et al. Note, however, that the choice of those parameters is difficult when one starts with a new experiment or
with a new type of outset data. One must explore countless
combinations to find the optimal parameters. We applied that
method to the two time series, s1 and s2, by fixing the window and the lag increment at one observation. We fixed
those two parameters because (a) they are less determinant
on the results, (b) we wished to simplify the explanations,
In the study of human movement, many researchers
attempt to understand the nature of intra- or interpersonal
coordination. For example, in a study on intrapersonal coordination, Haken, Kelso, and Bunz (1985) analyzed the interaction between the two fingers of one person; and in a study
on interpersonal coordination, Schmidt, Fitzpatrick, Bienvenu, and Amazeen (1998) analyzed the legs of two people
who sat facing each other. In an ecological situation such as
a tennis match that includes two players, studying the interaction is particularly difficult in the case of nonstationary
data such as the movement of their wrists. In this section,
we present a method called windowed cross-correlation
function (WCCF) that exceeds the classical temporal
method of cross-correlation by minimizing the effects of the
nonstationary properties. In the interest of clarity, we decided to present this method frequently used in motor control
studies in order to compare it with the CWT.
To illustrate the characteristics of the WCCF and the
CWT, we constructed a second synthetic signal. The second
signal, s2 (0.0–204.8 s), is generated from the properties of
only the first interval (0.0–51.2 s) of s1. Thus, s2 is purely
March 2006, Vol. 38, No. 2
WCCF
First, we introduce the classical cross-correlation function
that is often used in the study of motor interactions. Investigators use that traditional method to estimate the interaction
between two time series by exploring the relationships in
events. They proposed a method of cross-correlation to measure two vectors equal in the number of occasions of measurement by computing the Pearson product–moment of
those two time series. Cross-correlation gives us the strength
and the direction (sign of the correlation) of the interaction;
that is, it presents information on the phase relationship. It
gives the lag between two signals. The lag expresses the time
interval between events considered together, for example, the
time interval between the two time series. It is illustrated by
the gap of one series in relation to the other, and vice versa.
A lag of 10 ms shows that one person is 10 ms ahead when
compared with another person. That classical method of
cross-correlation seems to be very attractive, but there is one
essential condition: The observed signals must be stationary.
Boker, Xu, Rotondo, and King (2002) overcame the problem of potentially nonstationary data by calculating crosscorrelation in short intervals. As in the WFT computation,
windows are “swept over the whole data set” (Boker et al.,
p. 342). Those authors naturally called this method the windowed cross-correlation function. The cross-correlation
between the windows Wx and Wy is given as
r (Wx ,Wy) =
1
wmax
wmax
Wxi − µ(Wx )  Wyi − µ(Wy) 
i =1
SD(Wx )SD(Wy)
∑
,
(7)
151
J. Issartel, L. Marin, P. Gaillot, T. Bardainne, & M. Cadopi
and (c) we had no assumption on the selection of those parameters (see Boker et al.). The third parameter T is the time
lag. Tmax and Tmin are equivalent so that one can perform
the calculation in symmetry. We fixed the maximum lag at
2.5 s (50 points; see Figure 10). The fourth parameter is the
size of the window. We tested two window sizes: a narrow
window of 2.5 s (50 points; Figure 10A) and a wide window
of 10 s (200 points; Figure 10B). We set the window increment to one sample (i.e., 0.05 s). The WCCF figure is a twodimensional representation, as is the WT figure. In this
example, we focused on the correlation between .9–1.0
with the aim of exclusively showing the high interactions for
each lag. The results showed that in Interval 1 (0.0–51.2 s)
in Figure 10A and B, the correlation between s1 and s2 was
1.0 at Lag 0. That result is unsurprising because the two signals are identical in that interval. With the narrow window,
we can observe that the correlation alternated between high
and low correlations, with lags around 25, –25, 43, and –43.
Those periodicities appear with narrow windows and disappear when one increases the window (Figure 10B). Thus, the
augmentation of the window’s size introduces more observations that minimize the correlation. All the differences
between Figure 10A and B are the result of that property. In
the second interval (51.2–102.4 s), we increased the inter-
Interval
1
2
3
4
[0–51.2] [51.2–102.4] [102.4–153.6] [153.6–204.8]
Lag
A 50
0.9
0.2
0
1
–50
-0.2
0.9
0.2
0
50
Lag
B
–0.2
–1
–0.9
mediate frequency at the beginning and at the end of the
interval. One can see that the WCCF was more sensitive to
the low-frequency modulation (in the middle of that interval). The modulations on the low frequencies were more correlated than the modulation on the high frequencies. In the
third interval (102.4–153.6 s), one can see that the amplitude
modulation considerably influenced the results, because the
correlation coefficient diminished. At a certain level of the
amplitude’s decrease, the WCCF could not detect a high
level of interaction even if the frequencies were exactly the
same between the two signals. The effects were similar
regardless of the window size. In the fourth interval
(153.6–204.8 s), there were phase shifts of 30° on the highfrequency component and 135° on the low-frequency component. In Figure 10A, one can see that those phase shifts
induced periodicities on the correlation like those in the 2nd
interval. A lag at 0.19 s that expresses the phase shift by 30°
on s1 and a lag around 1.64 s that shows the phase shift of
135° can be observed. In Figure 10B, only the difference of
phase information at 135° can be seen because the wide window was precise only for the low frequency.
In simple situations, WCCF permits us to quantify the
level of interaction and the lag between two time series. A
limitation of that method arises from the sensitivity of its
four input parameters. The idea of subjectivity is strengthened when one notices that a simple modification in one
parameter such as the window size can radically change the
results. Moreover, that method seems limited because it is
impossible to obtain an optimal window with a multifrequency signal. Thus, the WCCF allows a partial alternative
for solving the problem of a nonstationary signal. In spite of
all those characteristics, however, that method appears to
solve only a few kinds of problems but forsakes the essential information contained in the time series, for example,
the relation of phase or the modification of frequencies. The
CWT method seems to allow one to solve those limitations.
CWT
Given two time series s1 and s2, with corresponding
transforms in the WT domain, S1 and S2, one can evaluate
interactions between s1 and s2 in that domain by computing
the cross-WT defined as (Daubechies, 1992)
0
(8)
S12 = S1 S2 ,
–50
0
50
100
150
200
Time (s)
FIGURE 10. Windowed cross-correlation function of synthetic signals s1 and s2 (A and B) obtained, respectively, for
a 2.5-s window (50 points) and a 10-s window (200 points).
The gray color represents the .9 to 1.0 correlation interval,
and the black color represents the –.9 to –1.0 correlation
interval. The lag increment was one observation.
152
where S2 is the complex conjugate of s2. Having chosen a
given mother function, one decomposes the two time series
on the same time scale basis. Any wavelet coefficient of the
time scale (frequency) representation of s1 expressed as one
spectrum and one phase (in the case of a complex-valued
mother function) can be compared with its equivalent for
s2, the product of the two spectra being a local measure of
the interaction between s1 and s2 at the given frequency and
the difference in phase between the two phases being a
Journal of Motor Behavior
Time–Frequency Anaylsis of Human Motor Behavior
direct measure of the local phase shift between s1 and s2 at
the given frequency.
One can generalize the statistical test, previously introduced in the WFT section, to the CWT. The confidence levels of the CWT can be derived from the square root of the
product of two chi-square distributions (Jenkins & Watts,
1968; Torrence & Compo, 1998). That combination of the
time and the frequency domains, with our goal of understanding the nature of the interaction between two time
series, permits us to access all the information contained in
the signals. The results of the CWT analysis (Figure 11)
permit us to access the interactions in terms of time, frequency, intensity, and difference of phase, all of which contribute to the legitimacy of that method (for comparison, we
present the results of the cross-windowed Fourier transform
in Figure 12 for the interested reader).
As mentioned earlier, in the computation of CWT between
s1 and s2, one assumes individual WT computation of s1 and
s2 on the same basis, that is, using the same Morlet mother
function that we used in the present work. Because s2 is purely stationary, the WT spectrum of s2 is characterized by three
horizontal bands corresponding to its three main frequency
components (not shown because the spectrum is the reproduction of the first time interval [0.0–51.2 s] of Figure 7B
over the full record [0.0–204.8 s]). One then obtains the CWT
spectrum of s1 and s2 by multiplying the square root of the
WT spectra of s1 and s2. Because the WT spectrum of s2 is
constant over the full record, the product of s1 and s2 is a
Interval
2
3
4
[51.2–102.4] [102.4–153.6] [153.6–204.8]
1
[0–51.2]
200
0
–200
–400
Amplitude
400
A
200
0
–200
Amplitude
400
B
–400
0
50
100
150
200
C
0.16
Frequency
(Hz)
0.44
0.08
D
0.16
Frequency
(Hz)
0.44
0.08
0
50
100
Time (s)
200
Phase (º)
Spectrum
Min
150
Max
–180
180
FIGURE 11. Cross-wavelet analysis of synthetic signals s1 and s2. (A and B) Representation of signals s1 and s2, respectively. (C) Cross-windowed Fourier transform spectrum. (D)
Phase difference between s1 and s2. Note that the high-frequency component of s2 was out
of phase by +30° (light gray) and that the low-frequency component of s2 was out of phase
by –135° (dark gray) in Time Interval 4.
March 2006, Vol. 38, No. 2
153
J. Issartel, L. Marin, P. Gaillot, T. Bardainne, & M. Cadopi
Interval
2
3
4
[51.2–102.4] [102.4–153.6] [153.6–204.8]
1
[0–51.2]
200
0
–200
–400
Amplitude
400
A
200
0
–200
Amplitude
400
B
–400
0
50
100
150
200
C
0.16
Frequency
(Hz)
0.44
0.08
D
0.16
Frequency
(Hz)
0.44
0.08
0
50
100
Time (s)
150
Phase (º)
Spectrum
Min
200
Max
–180
180
FIGURE 12. Cross-windowed Fourier transform (CWFT) of synthetic signals s1 (A) and s2
(B) generated with a cosine window of 25.6 s. Note the possibility of localizing the common
features of s1 and s2 (C) and quantifying their phase changes (D) in terms of time and frequency. As in WFT, the temporal resolution is determined by the window duration and the
frequency resolution depends on the analyzed frequency.
simple attenuation of the WT of s1 (Figure 11C). The CWT
of s1 and s2 is a smooth version of s1 as a result of the multiplication by (a) a local spectrum of zero magnitude in nonstationary intervals of the record of s1 and (b) a constant
spectrum in which s1 is stationary. The CWT spectrum may
have been different if s2 had been nonstationary, or simply if
we had set the intermediate frequency component of s2 to
zero in Time Interval 3 (102.4–153.6 s). In the latter case, a
gap (zero-magnitude spectrum) would have been observed in
the CWT spectrum for that interval and that frequency component. Whatever the nature of s1 and s2, one obtains the difference of phase (cross-phase) simply by measuring the local
signed difference between the local phase of s1 and the local
154
phase of s2. In the present example (Figure 11D), local phases
outlined by statistically significant spectra are in-phase (0° difference of phase), except in Time Interval 4 (153.6–204.8 s),
where the high- and low-frequency components of s1 are
phase shifted by +30° (light gray) and –135° (dark gray) with
respect to s2. Note that where the CWT spectrum is close to
zero, and thus is statistically not significant, computation of
the cross-phase is unstable and should not be done. As illustrated in that example, the CWT is extremely useful because
it provides objective (no presumption required) and simultaneous measures of (a) the intensity of the interaction and (b)
the difference of phase between the analyzed time series in
terms of time and frequency.
Journal of Motor Behavior
Time–Frequency Anaylsis of Human Motor Behavior
Applications to Real Data
In this section, we use a Morlet mother function with an
order of 8 to present an illustration of the WT and the CWT
on experimental data of human movement. Our aim in these
applications is to help the reader understand a scalogram
and interpret the results. We want to show that one can
make sense of any kind of data in experimental psychology
with that method. To do that, we chose not to present typical analyses such as Fitt’s law, interference effects, and
reaction times, among others. We conducted our analysis on
an extreme experimental situation that included nonstationary and nonlinear movements. We show that the method of
the CWT permits us to analyze and interpret complex
human motor behavior and complex coordination.
Two participants sat facing each other at a distance of 2 m.
Their right elbows were placed and fixed on the table. The
palms of their hands were facing forward, and their fists
were closed. We asked them (a) to move their wrists in the
sagittal plane by following a musical rhythm of drums and
(b) to be “tuned-in to each other,” which means that each
participant moved as a function of the other. Those were the
only instructions that we gave to the participants (they had
no previous experience in that kind of situation). We told
them nothing about the velocity, the amplitude, or the pattern
of the movements. We obtained data with the VICON 370
(Biometrics, Oxford, England) at a 50-Hz sampling rate. We
put one marker on the distal extremity of the ulna, one on the
extremity of the first forefinger phalanx, and two on the table
to acquire the angle between the horizontal and the fist. We
analyzed the movement on the y-axis.
The duration of the experiment was 2 min 15 s. To facilitate the explanation of the results and the legibility of the figures, we extracted two 32-s sequences: 10–42 s (Figure 13)
and 70–102 s (Figure 14). The movements of Participants 1
and 2, respectively, are represented on the y-axis of Figure
13A and B as a function of time. Figure 13C, E, and G represent the FT spectra of Participants 1 and 2 as well as the crossFT spectrum, respectively. Those figures permit the novice
reader of the WT scalogram to detect the main frequency
peaks present in the signal. Figure 13D and F represent the
WT spectrum of Participants 1 and 2, respectively. Figure
13H and I represent the cross-spectrum and the difference of
phase, respectively. With respect to Figure 13D, we can state
that there was one main significant frequency around 0.75 Hz.
There was frequency modulation of the main component
between 10 and 17 s and between 22 and 30 s at 0.57 Hz (note
the multifrequency components between 10 and 17 s): A significant frequency component at 0.3 Hz was associated with
the main modulated frequency component. Because we performed computations on the whole record and extracted the
discussed sequences from the complete results, those features
are real and cannot be associated with edge effects. In Figure
13F, we have relocated the same main significant component
(0.75 Hz) for Participant 2. One important difference, which
can be observed at the 23rd second (white arrow), characterizes a sudden augmentation of the frequency. That sudden
March 2006, Vol. 38, No. 2
augmentation was the result of an augmentation of the frequency component during one cycle. That brief change well
illustrates the sensitivity of the WT method. In Figure 13H,
which represents the cross-spectrum, one can see the significant interactions between the 2 participants that quantified our
visual interpretations. Thus, a main significant common frequency at 0.75 Hz between the 2 participants can be seen. An
effect of the sudden frequency modification at the 23rd second can be observed. The effect is visible on the cross-phase
(Figure 13I). That change produced a modification of the
phase between the 2 participants. Indeed, we can see that the
2 participants were initially in-phase (0°) and converged to the
out-of-phase (180°) after the 23rd second. That brief frequency modification delayed the 2 participants by one half-cycle.
We therefore can conclude that the CWT method is efficient
in the study of nonstationary signals and permits us to detect
small changes and to simultaneously analyze the temporal
evolution of the frequency and the difference of phase.
The second example (Figure 14) corresponds to the second
sequence (70–102 s) of the experiment that has just been presented. The second example shows some particularly complex and nonstationary signals with an association of multifrequencies and multiamplitudes. With the aim of
understanding all the characteristics of a scalogram, we now
discuss the results of the WT and CWT methods, essentially
focusing on other characteristics of the figures through an
explanation of the color intensity. We explain the scalogram
as a function of the shadings present in the significant areas.
As in Figure 13, the movements of Participants 1 and 2 are
represented on the y-axis as function of time in Figure 14A
and B, respectively. The WT spectra of each participant are
shown in Figure 14D and F, respectively. The cross-spectrum
is represented in Figure 14H, and the cross-phase in Figure
14I. As in the previous sequence, the FT spectra of Participants 1 and 2 and the cross-FT, respectively, are presented in
Figure 14C, E, and G. First, we can see in Figure 14D and F
the individual major effect, which is illustrated by the white
area in between 93–99 s; that area represents the highest local
spectrum of this sequence. The white area seems to be similar
for the 2 participants. Consequently, the CWT (Figure 14H)
displays that common component. At a high-frequency component around 2.4 Hz, one can see the major interaction in the
middle of the white area, with a decrease of that interaction to
the right and the left of the area. The relative phase of that
component shows exclusively an in-phase behavior (0°), represented by the gray color (see figure caption for the difference of phase). After that common stage of high-frequency/
high-amplitude activity, the 2 participants suddenly changed
their dynamics. Both signals in between 99 and 102 s are then
characterized by common components: an intermediate and a
low frequency. Even if the first part of the sequence (70–88 s)
is characterized by lower spectra, the frequencies are statistically significant and must be examined in further detail. In
that time interval, we observed that (a) Participant 1 performed a multifrequency movement with frequency components around 0.35, 0.50, and 0.75 Hz in between (74.0–85.5 s)
155
J. Issartel, L. Marin, P. Gaillot, T. Bardainne, & M. Cadopi
Movement in Frontal Plane (a.u.)
Participant 2
Participant 1
1
A
0.5
0
1
B
0.5
C
Frequency (Hz)
FT of Participant 1
0
2.0
D
1.0
0.87
0.75
0.57
0.5
0.3
0.25
10.0
15.0
22.5
30.0
Time (s)
37.5
42.0
FIGURE 13. Wavelet and cross-wavelet analysis of experimental data in between 10 and 42 s. (A and B) The
records of Participant 1 and Participant 2, respectively. (C and E) Normalized Fourier spectrum of (A) and (B),
respectively. A colorful spectrum with one main peak around 0.75 Hz can be seen in (C). A main peak around
0.75 Hz and a minor high-frequency peak around 2 Hz can be seen in (E). A last low-frequency component
below 0.3 Hz is also present. (D and F) Wavelet analysis of (A) and (B), respectively, help one to localize and
understand each of the components previously identified in the Fourier spectra. For example, the intermediate
frequency around 0.75 Hz in both signals fluctuated locally in between 0.5 and 1.0 Hz, explaining the noisy
nature of that peak in the Fourier spectra. Note also that the higher (~2 Hz) and lower (< 0.3 Hz) components
do not cover the full record but are, respectively, localized in time. Those intermittent behaviors explain the
weak amplitude of those peaks in the Fourier spectra. (G) The cross-Fourier spectrum and (H, I) cross-wavelet
spectrum and difference of phase of (A) and (B). Note that the common components, that is, intermediate frequency and low frequency in the first 4 s of the record, are highlighted. Whereas the 2 participants were initially in phase, they started to be phase shifted after the 27th second. a.u. = arbitrary unit.
and (b) Participant 2 performed a simpler movement with
low-frequency components around 0.35 and 0.5 Hz of high
amplitude in between (70–79 s). In the figure of the crossspectrum (Figure 14H), we were not surprised to see a strong
interaction only at the lower frequency component around 0.3
Hz. The difference of phase (Figure 14I) of that frequency
component showed us that the two records were locally in
antiphase at ±180°. Weaker interactions localized in common
time and frequency intervals (i.e., ~74–80 s; ~0.4 and 0.7 Hz)
were also detected and quantified in that interval. Finally, the
intermediate time interval (88–93 s) can be considered not statistically significant because the amplitude of both signals is
much lower in that short interval than in other portions of the
records. Investigation of that interval would then require a
zoom or a better estimate of the noise level, taken here as the
mean spectra (= background spectra).
Despite the complexity of the situation, the WT and, particularly, the CWT give us access to the nature of the inter156
actions between the 2 participants. The example of the real
data revealed that with the method of WT we are able to
analyze (and interpret) results even if the signals are particularly complex. Indeed, we observed that multifrequency
components alternated with a monofrequency component
and the amplitudes modulated within those frequencies.
Moreover, we observed some transitions of phase between
0° to 180°. The CWT permitted us to objectively show and
quantify the evolution of the structure of the motor behavior of the 2 participants.
Conclusion
We show in this article that independent analyses of a nonstationary signal, either on the single time basis or on the single frequency basis, provide global information. There is a
deficit of information because of those global (averaged)
analyses, which prevents us from describing the time-evolution of complex signals, a key step in the understanding of
Journal of Motor Behavior
Frequency (Hz)
E
Cross-FT
Frequency (Hz)
G
Spectrum
Max
Min
Difference of Phase (º)
–180
0
180
Frequency (Hz)
FT of Participant 2
Time–Frequency Anaylsis of Human Motor Behavior
2.0
F
1.0
0.87
0.75
0.57
0.5
0.25
2.0
H
1.0
0.87
0.75
0.57
0.5
0.3
0.25
2.0
I
1.0
0.87
0.75
0.57
0.5
0.3
0.25
10.0
15.0
22.5
complex movements in human motor behavior. If one’s objective is to understand the nonstationarity, then the time–frequency representation is a good tradeoff analysis with the
conjoint explanation of time and frequency domains. The
local information given by those methods (WFT and WT)
allows us to explain the frequency evolution as a function of
time by showing the information contained in signals in a new
two-dimensional representation (spectrogram and scalogram). We also demonstrated the superiority of the WT over
the WFT. The WT overcomes the limitations of the time–frequency basis of the WFT. Because of their local and multiscale properties, the WT revealed subtle changes in the signal
such as an amplitude modulation or a frequency modulation.
Those significant characteristics permitted us access to the
whole complexity of human motor behavior. We also demonstrated the superiority of the CWT on the WCCF in the study
of the interaction between two nonstationary signals. In the
last part of this article, the application of the WT on real data
March 2006, Vol. 38, No. 2
30.0
Time (s)
37.5
42.0
allowed us to illustrate the power of the WT on a real, complex situation. The method enables one to precisely detect the
signal properties in a very complex signal and is applicable to
a wide range of motor signals. Thus, this method seems well
suited for human movement studies, and it opens new perspectives of analysis in experimental psychology.
ACKNOWLEDGMENTS
Grants from the French Ministère de l’Education Nationale, de
l’Enseignement Supérieur et de la Recherche and from Enactive
Interfaces, a network of excellence (IST contract No. 002114), of
the Commission of the European Community provided support for
this research. We thank Idell Marin for critical discussion about
this work as well as for her help in ironing out last-minute English
language problems.
REFERENCES
Armand, M., & Minor, L. B. (2001). Relationship between timeand frequency-domain analyses of angular head movements in
157
J. Issartel, L. Marin, P. Gaillot, T. Bardainne, & M. Cadopi
Participant 1
A
0.5
0
1
Participant 2
Movement in Frontal Plane (a.u.)
1
B
0.5
C
D
Frequency (Hz)
FT of Participant 1
0
2.0
1.0
0.5
0.25
70.0
75.0
82.5
90.0
Time (s)
97.5
102.0
FIGURE 14. Wavelet analysis and cross-wavelet analysis of experimental data from 70 to 102 s (A)–(I)
are the same as in Figure 13. Note the complex nature of the record of Participant 1 in between 74–85 s
and the low-frequency dynamics of Participant 2 in that interval. For both participants, the time interval
93–101 s is first characterized by a high-frequency behavior that suddenly changes to lower frequency
behavior. As a main result, the cross-wavelet transform displays a common high-frequency component in
between 93–99 s and lower frequency components in the last part of that sequence (99–101 s). The 2 participants were mostly in-phase in that time interval (midgray color). a.u. = arbitrary unit.
the squirrel monkey. Journal of Computational Neuroscience,
11, 217–239.
Baraniuk, R. G., & Jones, D. L. (1993). Signal-dependent time–
frequency analysis using a radially gaussian kernel. Signal Processing, 32, 263–284.
Boker, S. M., Xu, M., Rotondo, J. L., & King, K. (2002). Windowed cross-correlation and peak picking for the analysis of
variability in the association between behavioral time series.
Psychological Methods, 7, 338–355.
Bullmore, E., Long, C., Suckling, J., Fadili, J., Calvert, G., Zelaya, F.,
et al. (2001). Colored noise and computational inference in neurophysiological (fMRI) time series analysis: Resampling methods in
time and wavelet domains. Human Brain Mapping, 12, 61–78.
Carevic, D. (2005). Adaptive window-length detection of underwater transients using wavelets. Journal of the Acoustical Society of America, 117, 2904–2913.
Chatfield, C. (1989). The analysis of time series. London: Chapmann & Hall.
Chui, C. K. (1992a). An introduction to wavelets. San Diego, CA:
Academic Press.
Chui, C. K. (1992b). Wavelets: A tutorial in theory and applications.
San Diego, CA: Academic Press.
Cohen, L. (1989). Time-frequency distributions—A review. Proceedings of the IEEE, 77, 941–981.
Daubechies, I. (1992). Ten lectures on wavelets. Philadelphia, PA: SIAM.
Daubechies, I., Grossmann, A., & Meyer, Y. (1986). Painless
nonorthogonal expansions. Journal of Mathematical Physics,
27, 1271–1283.
158
De Carli, F., Nobili, L., Beelke, M., Watanabe, T., Smerieri, A.,
Parrino, L., et al. (2004). Quantitative analysis of sleep EEG
microstructure in the time–frequency domain. Brain Research
Bulletin, 63, 399–405.
De Michele, G., Sello, S., Carboncini, M. C., Rossi, B., & Strambi,
S. K. (2003). Cross-correlation time–frequency analysis for multiple EMG signals in Parkinson’s disease: A wavelet approach.
Medical Engineering and Physics, 25, 361–369.
Feichtinger, H., & Gröchenig, K. (1988). A unified approach to
atomic decompositions via integrable group representations. In
M. Cwikel (Ed.), Lecture notes in mathematics (pp. 55–73).
New York: Springer.
Flandrin, P. (1988). Separability, positivity and minimum uncertainty in time–frequency energy distributions. Journal of Mathematical Physics, 39, 4016–4040.
Foufoula-Georgiou, E., & Kumar, P. (1994). Wavelet analysis and
its applications: An introduction. In E. Foufoula-Georgiou & P.
Kumar (Eds.), Wavelets in geophysics (pp. 1–43). San Diego,
CA: Academic Press.
Gabor, D. (1946). Theory of communications. Journal of the Institution of Electronical Engineering, 83, 429–457.
Gaudart, L., Crebassa, J., & Petrakian, J. P. (1993). Wavelet transform in human visual channels. Applied Optics, 32, 4119–4127.
Gilden, D. L., Thornton, T., & Mallon, M. W. (1995). 1/f noise in
human cognition. Science, 267, 1837–1839.
Haken, H., Kelso, J. A. S., & Bunz, H. (1985). A theoretical model
of phase transitions in human hand movements. Biological
Cybernetics, 51, 347–356.
Journal of Motor Behavior
Frequency (Hz)
E
2.0
F
1.0
0.5
0.25
Frequency (Hz)
FT of Participant 2
Time–Frequency Anaylsis of Human Motor Behavior
Cross FT
G
2.0
H
1.0
0.5
Spectrum
Max.
Min.
Difference of Phase (º)
–180
0
180
Frequency (Hz)
0.25
2.0
iI
1.0
0.5
0.25
70.0
75.0
82.5
Jenkins, G. M., & Watts, D. G. (1968). Spectral analysis and its
applications. San Francisco, CA: Holden Day.
Jobert, M., Tismer, C., Poiseau, E., & Schulz, H. (1994).
Wavelets—A new tool in sleep biosignal analysis. Journal of
Sleep Research, 3, 223–232.
Jones, D. L., & Baraniuk, R. G. (1995). An adaptive optimal-kernel
time–frequency representation. IEEE Transactions on Signal
Processing, 43, 2361–2371.
Karrasch, M., Laine, M., Rapinojac, P., & Krause, C. M. (2004).
Effects of normal aging on event-related desynchronization/
synchronization during a memory task in humans. Neuroscience
Letters, 366, 18–23.
Kumar, P., & Foufoula-Georgiou, E. (1997). Wavelet analysis for
geophysical applications. Reviews of Geophysics, 35, 385–411.
Li, C., Zheng, C., & Tai, C. (1995). Detection of ECG characteristic points using wavelet transforms. IEEE Transactions on
Biomedical Engineering, 42, 21–28.
Mann, S., & Haykin, S. (1995). The Chirplet transform. IEEE
Transactions on Signal Processing, 43, 2745–2761.
Morlet, J. (1983). Sampling theory and wave propagation. New
York: Springer.
Pajares, G., & De la Cruz, J. M. (2004). A wavelet-based image
fusion tutorial. Pattern Recognition, 35, 1855–1872.
March 2006, Vol. 38, No. 2
90.0
Time (s)
97.5
102.0
Sakowitz, O. W., Quiroga, R. O., Schurmann, M., & Basar, E.
(2001). Bisensory stimulation increases gamma-responses over
multiple cortical regions. Cognitive Bain Research, 11, 267–279.
Schmidt, R. C., Fitzpatrick, P. A., Bienvenu, M., & Amazeen, P. G.
(1998). A comparison of intra- and interpersonal interlimb coordination: Coordination breakdowns and coupling strength.
Journal of Experimental Psychology: Human Perception and
Performance, 24, 884–900.
Torrence, C., & Compo, G. P. (1998). A practical guide to wavelet
analysis. Bulletin of the American Meteorological Society, 79,
61–78.
Visser, E., Otsuka, M., & Lee, T. W. (2003). A spatio-temporal
speech enhancement scheme for robust speech recognition in
noisy environments. Speech Communication, 41, 393–407.
Zanone, P. G., & Kelso, J. A. S. (1992). Evolution of behavioral
attractors with learning: Nonequilibrium phase transitions.
Journal of Experimental Psychology: Human Perception and
Performance, 18, 403–421.
Submitted March 2004
Revised July 15, 2005
159