Sequential Tests: a Tool for GW Detection
M. Longo, S. Marano , V. Matta, and I. M. Pinto
Dept. of Electrical and Information
Engineering, University of Salerno
Via Don Melillo, 84084
Fisciano (SA), Italy
{longo, marano, vmatta}@unisa.it
Dept. of Engineering,
University of Sannio at Benevento
C.so Garibaldi 107, I-82100
Benevento, Italy
INFN and LSC
[email protected]
Physics Department , University of Rome, Rome– January 22-29, 2010
A question
• How to detect a (GW) signal (say, a sinusoid) embedded
in noise (say, AWG)?
The well-known recipe
• Take your n samples of data
• Implement the optimal Likelihood Ratio Test
V. Matta
Sequential Tests: a Tool for GW Detection
2
A question
• However…Some GW signals (e.g., continuous sources) can be
observed for a very long (virtually infinite) time
How many samples should I collect?
• Fix in advance the number of samples n, based upon the
required performances and/or computational power
• Implement the optimal Likelihood Ratio Test
V. Matta
Sequential Tests: a Tool for GW Detection
3
Is this the best answer?
V. Matta
Sequential Tests: a Tool for GW Detection
4
A question
• However…Some GW signals (e.g., continuous sources) can be
collected for a very long (virtually infinite) time
How many samples should I collect?
• Do NOT fix in advance the number of samples n
• Still use a Likelihood Ratio Test?
V. Matta
Sequential Tests: a Tool for GW Detection
5
Motivations
• Some of us (the speaker, for instance) come from the Signal
Processing Community
• Please, assume that I don’t know anything about GWs (…and you will
be not so far from the truth)
• We have gained a reasonable expertise in Sequential Detection
SP Community
GW Community
• Stimulating interaction with people in the GW Community suggested
that the Sequential paradigm might fit the continuous GW problem
V. Matta
Sequential Tests: a Tool for GW Detection
6
Motivations
What you won’t find in this talk
• You won’t find a Pulsar (Alas…)
• You won’t find a “turn-the-handle” signal processing solution
for GW detection
What you will find in this talk
• You will (hopefully) find that sequential analysis provides a
suitable framework for some GW detection problems
• You will find a number of problems which call for mutual
interaction between the SP and the GW communities
V. Matta
Sequential Tests: a Tool for GW Detection
7
Introduction
proper framework: statistical hypothesis testing
Let’s start with the simplest (toy) case of a known DC-level
hidden in independent noise samples
available data
signal
H0 : x k    wk
H1 : x k  w k

V. Matta
noise waveform
Sequential Tests: a Tool for GW Detection
8
Introduction
optimality criterion:
NP (Neyman-Pearson)
Fix the false alarm probability Pf to a tolerable limit
Pf = Pr{ signal presence is declared… but you are wrong }
and maximize the detection probability Pd
Pd = Pr{ signal presence is declared… and you are right }
Bayesian, min-max or other formulations clearly possible
V. Matta
Sequential Tests: a Tool for GW Detection
9
Introduction
1) Compute the log-likelihood ratio (LLR) L(r) of the data
2) Compare it to a threshold  ( is a function of the given Pf)
   choose H1
L(x)
   choose H0

V. Matta
Sequential Tests: a Tool for GW Detection
10
Sequential Tests: Basics
Wald’s viewpoint [Wald,1947]
• Fix both Pf and Pd as you like
• Minimize the (average) number of samples to achieve
the desired performance
OPTIMAL STOPPING TIME to end the test
V. Matta
Sequential Tests: a Tool for GW Detection
11
Sequential Tests: Basics
Sequential Probability Ratio Test (SPRT)
• Still based on (running) LLR L(r)
• Involves two thresholds 0 and 1
r = [r1,…rk]
 1 
choose H1

k
L(x ) ( 0 , 1 )  take one more sample
  
choose H0
 0

V. Matta
Sequential Tests: a Tool for GW Detection
12
Detection Statistic
A pictorial view of Sequential Tests
1
Time
0
(RANDOM)
Stopping Time
V. Matta
Sequential Tests: a Tool for GW Detection
13
SPRT optimality [Wald & Wolfowitz, 1948 ]
• SPRT tries to “understand” from the specific realization
the appropriate time N to stop testing.
• Appropriate means that the desired Pd and Pf are
achieved
• SPRT minimizes E[N] (average stopping time) among
all sequential tests achieving the same Pd and Pf
• The fixed-sample-size (FSS) LRT is a special case of
sequential procedure with vanishing VAR[N]
That’s why the SPRT is known to outperform the LRT
V. Matta
Sequential Tests: a Tool for GW Detection
14
SPRT optimality: criticism
The average stopping time is minimized. What about the single realization?
• The CDF of the stopping time helps to quantify the variability around the
expectation. Typically, significant gains w.r.t. the LRT arise
• In practice you run tons of tests: an average criterion gives you relevant
savings, thanks to the Law of Large Numbers
• You may occasionally experience the Long Run: this is solved in practice
by resorting to a truncated SPRT*
* S. Tantarantana, H.V. Poor, “Asymptotic Efficiencies of Truncated Sequential Tests”,
IEEE Transactions on Information Theory, Vol. 28, No. 6, pp. 911-923, 1982.
V. Matta
Sequential Tests: a Tool for GW Detection
15
SPRT : relevant formulas
Average Stopping Times
Easy threshold setting
Pd 1  (1 Pd ) 0
E[N H1 ] 
 f (x) 
E log 1
H1
 f 0 (x) 
1 Pd
 0  log
1 Pf
1  log
Pd
Pf
Pf 1  (1 Pf ) 0
E[N H 0 ] 
 f (x) 
E log 1
H 0 
 f 0 (x) 
Approximations hold for “small” (vanishing) error probabilities and/or
“small” (vanishing) SNR
That’s OK for the GW 
regimes of interest!!!
V. Matta
Sequential Tests: a Tool for GW Detection
16
Putting pieces together…
E[N H1] 
E[N H 0 ] 

Db Pd Pf

D f1 f 0 
Db Pf Pd 
D f 0 f1
where we introduced the informational measures

 f (x) 
f (x)
D( f1 || f 0 )  Elog 1
H1  f1(x)log 1
dx
f 0 (x)
 f 0 (x) 
Kullback-Leibler distances
Pd
1 Pd
Db (Pd || Pf )  Pd log
 (1 Pd )log
Pf
1 Pf

V. Matta
Sequential Tests: a Tool for GW Detection
The Test Exponents
• In the SPRT case, the error probabilities scale exponentially fast with
the number of samples
E[ N H 0 ]

1 Pd
e
E[N H 1 ]
Pf

e
-E[N H 0 ]D  f 0 f1 
-E[N H1 ]D  f1 f 0 
Best attainable exponents

Sequential
V. Matta Tests: a Tool for GW Detection
18
Quantifying advantages
• OK, using an optimal procedure is the right thing to do
• On the other hand your question is legitimate:
How much can we gain?
• The exact gain depends upon the specific detection problem
under consideration
• For scholarly cases (with small SNR), gains up to an order
of magnitude are the rule*
• S. Tantarantana, J.B. Thomas, “Relative Efficiency of the Sequential Probability Ratio Test in
Signal Detection”, IEEE Transactions on Information Theory, Vol. 24, No. 1, pp. 22-31, 1978.
V. Matta
Sequential Tests: a Tool for GW Detection
19
Can we work only with a perfectly known, constant signal ?
IF YES==> You’ll tell me “Please QUIT and go out.
It is recommended to run in order to avoid heavy
objects falling on your head.”
ELSEIF NO==>You’ll tell me “Please QUIT and go out.
Running not needed.”
END
V. Matta
Sequential Tests: a Tool for GW Detection
20
A more realistic case (non i.i.d.)
H 0 : x k  sk  w k
H1 : x k  w k
• Design formulas are almost identical
• Exploiting
 Martingale Theory [see, e.g., Doob],
performance analysis is straightforward for a broad
class of signals of interest (including Dopplermodulated ones)
• Theoretical studies about SPRT asymptotic optimality
for the non i.i.d. case are available*
* A. G. Tartakovsky, “Sequential Composite Hypothesis Testing with Dependent
Nonstationary Observations”, Prob. Inform. Transm., Vol. 17, No. 1, pp.18-28, 1981.
V. Matta
Sequential Tests: a Tool for GW Detection
21
A more realistic case (Unknown SNR)
• Resort to Locally Optimum Detection for a
nonparametric approach…
• Elegant extensions to the Sequential case provided by
Lai*, based upon functional versions of the Central
Limit Theorem, and SPRT between Wiener processes
T.L. Lai, “Pitman efficiencies of Sequential Tests and Uniform Limit Theorems in
Nonparametric Statistics”, The Annals of Statistics, Vol.6, No.5, pp. 1027-1047,
1978.
V. Matta
Sequential Tests: a Tool for GW Detection
22
Challenges for GW detection
Does SPRT give only advantages?
All done ?
Obviously, no!
In many cases SPRT is more difficult to analyze than
the Likelihood Ratio Test
Presence of unknown parameters: no simple
Generalized LRT (GLRT) analogue
V. Matta
Sequential Tests: a Tool for GW Detection
23
Open problems
• There exists no assessed GLRT-counterpart of the
Sequential Probability Ratio Test
• Lack of a standard design criterion for these cases
• Optimality properties still to be formally understood
• Application to a real-world GW model
• ….
V. Matta
Sequential Tests: a Tool for GW Detection
24
The SPRT Bank*
• For instance, consider an almost harmonic signal with unknown
amplitude, frequency and phase
• Each chunk of m samples is transformed via an FFT
• Bank of incoherent stages combined sequentially
• The loss incurred by the incoherent approach is mitigated by the
SPRT gains
* S. Marano, V. Matta, P. Willett, “Sequential Detection of Almost-Harmonic Signals”,
IEEE Transactions on Signal Processing, Vol.51, No.2, pp. 395-406, 2003.
V. P. Dragalin, A. G. Tartakovsky, V. V. Veeravalli, “Multihypothesis Sequential Probability Ratio Tests.
Part I: Asymptotic Optimality, IEEE Transactions on Information Theory, Vol. 46, No. 4, pp. 1366-1383, 2000.
V. Matta
Sequential Tests: a Tool for GW Detection
25
The SPRT Bank
f1
f2
fK
0
1
0
0
0
0
0
0
1
0
1
0
1
0
1
1
N2
There’s no signal
NK
0
There’s no signal
Computational power is
Still running
re-allocated dynamically
Time axis (No. of samples)
V. Matta
Sequential Tests: a Tool for GW Detection
26
Another issue: Change Detection
A number of related problems can be of interest for GW community
Consider the opposite situation of a signal appearing
suddenly , which should be detected as quickly as possible
f 0 (x) : x1, x 2 , x 3 ...x n 0
Change in distribution
x n 0 1, x n 0 2 , x n 0 3 ...
f1 (x) :
V. Matta
Sequential Tests: a Tool for GW Detection
27
A pictorial view of Page’s test
Detection Statistic
Change of
Distribution
Stopping
Time

Change
Detected
n0
Time
Detection Delay
V. Matta
Sequential Tests: a Tool for GW Detection
28
Page’s test
Page’s test philosophy in a nutshell
• Minimize the average detection delay D
• Constrained on the average time interval between false
alarms R
Optimal CUmulative-SUM Detection Statistic

f1 (x n ) 
Tn  max 0,Tn1  log

f 0 (x n ) 


T0  0
Average Detection Delay
Easy threshold setting
log(1/R)
E[D] 
D( f1 || f 0 )
1
  log
R
V. Matta
Sequential Tests: a Tool for GW Detection
29
Conclusions
• Sequential analysis is a suitable framework for some
issues in GW detection experiments
• Some live problems in sequential detection are of
interest for the GW Community
• Cross-breeding approach much in order to make things
work with realistic GW waveforms and noises
V. Matta
Sequential Tests: a Tool for GW Detection
30