Efficient Likelihood Evaluation and Dynamic Gaussian Selection for HMMbased Speech Recognition
Jun Cai1,2, Ghazi Bouselmi1, Yves Laprie1, Jean-Paul Haton1
1
Groupe Parole, LORIA-CNRS & INRIA, BP 239, 54600 Vandoeuvre-les-Nancy, France
2
Dept. of Cognitive Science, Xiamen Univ., 361005 Xiamen, China
{Jun.Cai, bousselm, Yves.Laprie, [email protected]}
Abstract — LVCSR systems are usually based on continuous density HMMs, which are typically
implemented using Gaussian mixture distributions. Such statistical modeling systems tend to operate
slower than real-time, largely because of the heavy computational overhead of the likelihood
evaluation. The objective of our research is to investigate approximate methods that can substantially
reduce the computational cost in likelihood evaluation without obviously degrading the recognition
accuracy. In this paper, the most common techniques to speed up the likelihood computation are
classified into three categories, namely machine optimization, model optimization, and algorithm
optimization. Each category is surveyed and summarized by describing and analyzing the basic ideas
of the corresponding techniques. The distribution of the numerical values of Gaussian mixtures
within a GMM model are evaluated and analyzed to show that computations of some Gaussians are
unnecessary and can thus be eliminated. Two commonly used techniques for likelihood
approximation, namely VQ-based Gaussian Selection and Partial Distance Elimination, are analyzed
in detail. Based on the analyses, a fast likelihood computation approach called dynamic Gaussian
selection (DGS) is proposed. DGS approach is a one-pass search technique which generates a
dynamic shortlist of Gaussians for each state during the procedure of likelihood computation. In
principle, DGS is an extension of both techniques of Partial Distance Elimination and Best Mixture
Prediction, and it does not require additional memory for the storage of Gaussian shortlists. DGS
algorithm has been implemented by modifying the likelihood computation procedure in HTK 3.4
system. Experimental results on TIMIT and WSJ0 corpora indicate that this approach can speed up
the likelihood computation significantly without introducing apparent additional recognition error.
Keywords —Gaussian selection, fast likelihood computation, hidden Markov models, speech
recognition.
I. INTRODUCTION
Most state-of-the-art large vocabulary continuous speech recognition (LVCSR) systems are based on
hidden Markov models (HMM). To obtain a high level of accuracy, HMM-based LVCSR systems
typically use continuous density HMMs (CDHMMs) [1] as the underlying technology to perform
acoustic modeling of speech signals. In such systems, the output likelihood of a HMM state S for a
given observation vector, xn, can be expressed as a weighted sum of probabilities:
p  xn S

  k pk  x n 
M
(1)
k 1
Here, M represents the number of mixture components for state S;  k is the mixture weight for kth
M
component, which satisfies  k  0 and   k  1 . In Eq. (1), each of the probability terms is usually
k 1
expressed by a multivariate Gaussian density:
pk  x n  
1
2 
N/2
Σk
1/ 2
 1

T
exp    x n  μk  Σ k1  x n  μk  
2


(2)
where N is the dimensionality of the feature vector, μk and Σ k are the mean vector and covariance
matrix for the kth Gaussian density in state S. With the probabilities defined in Eq. (2), the state model
in Eq. (1) becomes a Gaussian mixture model (GMM) with M Gaussian components. In practice,
modeling states with diagonal covariances are preferred over the use of full covariance matrices for
computational and data sparseness reasons. Therefore, Eq. (1) becomes:
p  xn S

M

k 1

k
2 N / 2    kq2 
N
 q 1
1/ 2

 1 N xnq   kq 2 
exp  

2
 kq

 2 q1


M
 1 N xnq   kq
  Z k exp  
2
 kq
k 1
 2 q 1

2
 M
   N  x n ; Z k , μk , Σ k
 k 1

(3)
where Zk is a constant for each Gaussian. Since all terms that comprise Zk are known prior to
recognition, it can be computed in advance. In order to compute efficiently and to avoid underflow,
probabilities are computed in the log domain. Therefore, the log likelihood can be expressed as:

log p  x n S
 


2
M

1 N xnq   kq 
logadd  logZ k   

2
2 q 1
 kq
k 1


(4)
Here, the function logadd  is defined as follows:
M
k 1
M
M

logadd wk   log   expwk 
k 1
 k 1

(5)
In a typical HMM-based LVCSR system, the number of model states ranges from 2000 to 6000, each
of which is a weighted sum of typically 8 to 64 multidimensional Gaussian distributions as in Eq. (3).
For each input frame, the output likelihoods should be evaluated against each active state. Therefore,
the state likelihoods estimation is computationally intensive and takes about 30% to 70% of the total
recognition time [2, 3]. To derive high level of recognition accuracy, even more Gaussians can be used
and thus the percentage of the recognition time used in Gaussian evaluations could be higher. This
kind of likelihood-based statistical acoustic decoding is so time-consuming that it is one of the most
important reasons why the recognition is slow. Some LVCSR systems might even decode speech
several times slower than real time [4]; that is to say, these systems are not practical for most
spontaneous applications, such as man-machine dialogue. Therefore, it is necessary to develop
efficient techniques in order to reduce the time consumption of likelihood computation without a
significant degradation of recognition accuracy.
A wide variety of techniques based on different strategies have been devised to speed up the likelihood
computation. In this paper, these techniques are classified into three categories with respect to three
basic tactics: machine optimization, model optimization, and algorithm optimization. Each category is
surveyed by describing and analyzing corresponding techniques. In the category of algorithm
optimization, the most popular are VQ-based Gaussian selection techniques [2, 3, 5-10]. A typical VQbased Gaussian selection technique can lead to additional memory requirements for the storage of
cluster centroids. To overcome this problem, we propose an alternative scheme, dynamic Gaussian
selection (DGS), based on the partial distance elimination (PDE) framework [11-13]. DGS aims at
maintaining recognition accuracy with no additional memory requirement.
The paper is organized as follows. The classification and the review of the three tactics of optimization
are presented in Section II. In Section III, the Gaussian selection techniques are reviewed and analyzed.
Section IV describes a nearest neighbor approximation technique based on PDE. In Section V, the
proposed DGS technique is presented in detail, which uses the extended PDE method to limit the log
likelihood evaluation of each GMM to the computation on a small set of dynamically selected
Gaussian components. This technique is evaluated with English continuous speech corpora TIMIT and
WSJ0. Experimental results are presented in Section VI. It is concluded in Section VII that DGS is an
efficient technique for fast likelihood evaluation, and combining DGS with other optimization
techniques can give rise to satisfactory real-time performance.
II. BASIC TACTICS TO SPEED UP LIKELIHOOD COMPUTATION
For designing an ASR software, the core issues can be basically divided into two aspects: the design of
models and the design of algorithms. All the design of models and algorithms should be coded in a
programming language and deployed on a computing platform. Therefore, the design and development
of the likelihood estimation module in an HMM-based LVCSR system can be divided into three parts
— the design of HMM models, the design of algorithms, and the implementation and deployment of
the software on a certain platform. In such a breakdown scheme, the achievement of fast likelihood
evaluation relies on three fundamental design decisions, which we call tactics. Since the focus here is
to speed up the likelihood calculation, a tactic is a design decision that uses a technique with respect to
one of the three parts in order to optimize the time response of the likelihood computation. Specifically,
these three tactics can be called model optimization, algorithm optimization, and machine optimization,
as shown in Fig 1.
Fig. 1 Three Tactics for Fast Likelihood Computation
Every technique for fast likelihood computation can be categorized as one of these three optimization
tactics. For example, semi-continuous or tied-mixture HMM [1] is a technique of model optimization,
and various Gaussian selection techniques belong to algorithm optimization tactic. Using parallel
computing to achieve fast calculation of likelihoods is actually putting machine optimization tactic into
practice. So far, most implementations of fast likelihood evaluation have used only one or two
techniques belonging to the same optimization tactic. Since different tactics deal with different aspects
of LVCSR system design, several optimization techniques at different tactic levels could be integrated
in application. For instance, based on optimized models, we can design and adopt techniques for
algorithm optimization to decrease the algorithmic complexity, and can furthermore implement the
software with techniques for machine optimization. With this kind of integration, the time overhead
for likelihood evaluation could be reduced to the greatest possible extent.
A. Techniques for Model Optimization
Model optimization tactic concerns techniques of GMM complexity reduction. Usually, in a highly
accurate HMM-based acoustic model, a large number of GMMs are used, as well as a large number of
Gaussians in each GMM. The number of free parameters in such a system can be very large. A model
with too many free parameters suffers from the problem of a large time complexity of likelihood
computation. However, there are similar parameters at different levels among all the GMMs within a
system. Thus, it is desirable to tie some close parameters together in order to reduce the models’
degree of freedom. By tying two or more groups of parameters from different models at the same level,
we usually consider these groups as a unique entity. For instance, N similar Gaussians belonging to N
different GMMs respectively can be tied together and represented by a single Gaussian. Those tied
Gaussians will share the same probability density and the likelihood computation will be performed
only on their unique representative.
There are basically three widely used parameter-tying techniques to achieve model complexity
reduction, each corresponding to a specific granularity, as shown in Fig. 2. The first one is to cluster at
the model level; specifically, to decrease the degree of model complexity by clustering HMMs. A
conventional method to perform such clustering is to merge similar phonetic models together [14]. The
disadvantage of this technique is that it may force probability distributions with quite different shapes
to be mixed and thus degrade the decoding accuracy. The second way is to cluster at the granularity of
state rather than of the entire model [15-17]. Each cluster represents a set of similar Markov states and
is called a senone. Tying parameters at the Markov state level can keep dissimilar states of different
models apart while other corresponding states are merged, thus leading to better parameter sharing
than in clustering HMMs. Going down the schemes in Fig. 2, parameter-tying can be achieved at a
finer level of Gaussians. The so-called semi-continuous HMM (SCHMM) [18] technique is a typical
example of such parameter sharing. It is a full tying scheme in which the Gaussian mixture
components are tied together across all the models to form a set of shared bases. Several variants of the
SCHMM have been proposed [19, 20]. One is to cluster Gaussian components of all well-trained
GMMs in order to form a set of shared components which are called genones [19]. The main
difference between this genone scheme and the SCHMM scheme is that even though a shared set of
genones is formed, it is not necessary for a genone to be shared among all the GMMs. So, this genone
technique can be considered as an intermediate scheme between fully continuous HMM and SCHMM
schemes.
Fig. 2 Three Granularities of Parameter-Tying
In essence, model optimization at whatever level is an approximate computation. The finer the
granularity of parameter-tying is, the smaller the approximation error could be. Even parameter-tying
at the granularity finer than a Gaussian component is feasible. One of such tying techniques is the subvector clustering, also known as subspace distribution clustering [21, 22]. This method breaks down
each multi-dimensional Gaussian component into several fragments, each of which corresponding to a
chunk of the observation vector. The fragments are clustered piecemeal over all states of all acoustic
models. Since the clustering granularity is finer, the quantization errors can be much smaller than those
of clustering at entire Gaussian component level.
B. Techniques for Machine Optimization
Machine optimization tactic focuses on taking advantage of modern computer architecture to facilitate
fast speech recognition. In contrast to other tactics, machine optimization normally introduces no
accuracy degradation. With the development of parallel computing technology and the availability of
parallel computer hardware, implementing speech recognition with parallel algorithms becomes a
promising technique of machine optimization tactic to achieve real-time performance. In the context of
likelihood computation, likelihoods of a certain observation vector should be computed successively at
different levels ranging from elements in a Gaussian, Gaussian components, model states, to HMMs.
For an input frame, computations on an object at a certain level are normally independent of the
computations on other objects at the same level. Therefore, the likelihoods can be computed in parallel
within each of these levels. A commonly employed method to achieve such parallel computation is
utilizing MPI API [23] to have several processes to compute likelihoods on a set of processors. Since
multi-core processors, such as Intel Pentium™ Dual Core Processor and AMD Athlon™ 64 X2 Dual
Core Processor become popular, even on modern PC platforms the likelihood computation can be
implemented with thread-level parallelism to some extent.
Another feasible method to achieve parallelism is to use SIMD (Single Instruction, Multiple Data)
instructions to handle the data manipulation in likelihood computation [24]. Most modern processor
architectures (e.g., Intel’s SSE and AMD’s Enhanced 3DNow!) provide SIMD instructions to achieve
data level parallelism — loading a number of data at once and applying the same instructions to all of
them in one operation, thereby speeding up algorithms based on vector or matrix manipulation. An
application that may take advantage of SIMD is the one where the same mathematical operations are
performed on a large number of data points, which is a common operation in many multimedia
applications. By inspecting Eq. (3), we can see that likelihood computation is such an application that
it consists of a series of subtractions and multiplications iterated over elements of Gaussians. So,
SIMD instructions can be fully employed in coding the iterative Gaussian computation. In addition,
CPU cache manipulations, such as cache prefetch [25, 26], can be used to increase the data access
efficiency and therefore to further improve the real time performance.
Since CPU with SIMD units and increasing volume of cache are becoming widely used in PC systems,
exploring how to use SIMD operations and cache manipulations to carry out data-level parallelism and
to achieve real-time performance is a promising research topic in developing speech recognition
software. SIMD and cache prefetching techniques have been successfully applied in a wide range of
video games. By drawing on the experience in developing these applications and applying optimized
math libraries1 that use SIMD instructions, it is practical to modify the sequential computing modules
for likelihood evaluation in existing speech recognition software to achieve SIMD-based data-level
parallelism [26]. A technical issue related to the use of SIMD instructions is the address alignment of
the Gaussians data in memory. Indeed, most CPUs with SIMD units require the data loaded into its
registers (or the data output from registers to memory) to be located at aligned addresses which are
multiples of a certain granularity, which is generally the size of the registers. When this condition is
not satisfied, the load operations would require special access to the processor’s cache lines and to the
memory, resulting in a heavy time overhead for the access. Therefore, a challenging task in performing
likelihood evaluation with SIMD operations is that mean and variance vectors should be aligned
properly in memory in order to maximize the benefits of SIMD parallelism.
C. Techniques for Algorithm Optimization
The focus of algorithm optimization is on simplifying algorithm for likelihood computation. Most of
the techniques of this category use some kind of approximation algorithm to replace the full likelihood
computation. The criterion for deciding whether an approximation algorithm can be adopted is usually
that the relative degradation of recognition accuracy is less than 5%. Up to now, the majority of fast
likelihood evaluation techniques belong to this category. Following the categorization scheme in [27],
the GMM computation can be broken down into four different layers, where lower layers provide
support to upper ones. Each individual fast computation algorithm is associated with one specific layer.
This four-layer categorization scheme is shown in Fig. 3.
Fig. 3 Four-Layer Categorization Scheme of Algorithm Optimization Techniques
1
Intel and AMD both provide such optimized math libraries, and open source alternatives such as libSIMD and SIMDx86 are
already available.
Frame-Layer algorithms use some criteria to decide whether all GMM likelihoods should be computed
or skipped for each frame [27]. For a skipped frame, the likelihoods are copied from those of the most
recently computed frame. In the design of these algorithms the following assumption is made:
neighboring input frames are correlated, and in some speech segments consecutive input vectors are
close to each other in the acoustic space. The simplest example of the algorithm is to compute
likelihoods only on every other frame, which is called simple down-sampling. A more complex
scheme is called VQ-based down-sampling technique in which a feature vector VQ codebook is
generated from all means of GMMs in a set of trained acoustic models, based on some clustering
method. In decoding, every input vector is quantized first according to the codebook. A frame is
skipped only if its feature vector is quantized to the same codeword as that of the previous frame.
Although skipping some frames’ likelihoods in both the down-sampling schemes can effectively
eliminate a part of computation, it causes problems. With simple down-sampling, half of the input
frames are compulsorily skipped, leading to a significant loss of recognition accuracy. With VQ-based
down-sampling, if the codebook is sufficiently large, we can guarantee that no considerable
degradation of accuracy will be introduced in. But in this case, there is a little chance for an input
frame to be quantized into the same codeword as that of the previous one. Therefore the performance
improvement by using down-sampling schemes would be very limited.
GMM-layer algorithms speed up the computation by ignoring some GMMs in the likelihood
computation for each computed frame. A typical technique of this category is the context-independent
HMM-based GMM selection used in Julius [28, 29]. Briefly, the technique is a two-pass search. For a
phone, the algorithm uses a simple context-independent monophone HMM and its corresponding
triphone models. It is assumed that all triphone models with the same center phone have the same
number of states as the corresponding monophone model, and that they are trained with the same data.
The monophone models could be rough with small numbers of Gaussians while corresponding
triphone models are precise with large numbers of Gaussians. All monophone states are evaluated first
for each input frame, and a beam is applied in order to determine the k-best states. Then in the second
pass, only Gaussian mixtures of the triphone states that correspond to the k-best monophone states are
computed in detail. For triphone states whose corresponding monophone states are not selected in the
k-best, likelihoods are approximated by assigning the likelihoods of the monophone models. This twopass search scheme is highly effective in reducing GMM computation. According to some empirical
studies [27], this context-independent HMM-based two-pass search outperforms other existing speedup techniques within the constraint of 5% accuracy degradation in terms of that it can bring the best
speed enhancement in the whole recognition procedure, including both GMM computation and
searching. For likelihood computation, there can be an 80% reduction in time. However, because
likelihoods of some triphone states are approximated with the results evaluated in the first pass,
triphones corresponding to a certain unselected monophone will be assigned the same likelihood when
they are fed into the search module. With a tight beam in the first pass, the likelihoods of many
triphone states tend to be the same and the range of likelihood values becomes narrow. As a
consequence, the search will be more computationally intensive and beam pruning becomes less
effective. Research shows that this problem can be solved by tightening the Viterbi beam.
Another commonly used approximation method is to simplify likelihood computation in the Gaussian
layer. Briefly, there are two types of Gaussian-layer algorithms. The most common type is the wellknown Gaussian selection which attempts to omit some Gaussians in the computation of the mixture
model. Another one is the nearest-neighbor approximation method which tries to take the maximum
Gaussian as the state likelihood rather than summing across all Gaussian components of the mixture
model. Since many different algorithms have been proposed in this sub-category and both Gaussian
selection and the nearest-neighbor approximation constitute the basis of the proposed DGS algorithm,
these two methods will be elaborated in the following sections.
Some fast likelihood computation techniques utilize the characteristics of the feature elements to
change the iterative computation of weighted Euclidean distance in Eq. (3). We categorize this kind of
techniques as the element-layer algorithms. One such algorithm is the feature pruning [30, 31]
technique in which only a fraction of feature elements are evaluated to estimate the likelihoods. We
can notice that if any one of the 1-dimensional Gaussian factors in Eq. (3) is very small, the full-
dimensional Gaussian likelihood will be very small too. Therefore, rather than always computing the
likelihood of the full Gaussian component, we compute it only in the case when the first 1-dimensional
Gaussian factor is large enough. If the first sub-dimensional Gaussian is small, since the entire
Gaussian component will have a low probability value, it can be simply approximated by using a
flooring value or a computationally inexpensive function. This technique can be easily extended to
multi-iteration cases in which the recursion in Eq. (3) stops and the approximation is applied when the
current sub-dimensional Gaussian falls below a threshold. Furthermore, empirical studies show that
some feature elements are “prominent” in the sense that the sub-dimensional Gaussians on these
feature elements usually have smaller values than those of others. Therefore, reordering the feature
elements in order to compute on “prominent” feature elements first in Eq. (3) will further improve the
speed of likelihood computation. This feature element reordering (FER, aka feature component
reordering or FCR) technique can also be used to complement Gaussian-layer algorithms such as the
nearest-neighbor approximation to further reduce the likelihood evaluation time [12, 13].
III. VQ-BASED GAUSSIAN SELECTION
Analogous to series expansions used to approach complex functions, the Gaussian mixture model in
Eq. (3) is actually an approximation mechanism to compute various probability distributions. Usually,
for a given observation vector, some Gaussians in the mixture make a “heavy” contribution to the
likelihood, while others contribute little. To illustrate the distribution of the logarithm of Gaussian
components corresponding to an observation vector, we have built a recognition system with 32Gaussian HMMs on TIMIT with HTK 3.4. The log likelihoods of 50 monophone states for emitting
the first frame of the consonant “p” are computed, 4 typical ones are depicted in Fig. 4. In each
diagram, the logarithm of 32 Gaussians (C1, C2, …, C32) in a certain GMM are shown in bar charts,
while the log likelihood of the whole GMM as well as the maximum log Ci (i = 1, 2, …, 32) are
presented at the bottom of each diagram. Furthermore, a phone level recognition experiment has been
performed to make a statistical analysis of the contribution of different Gaussians to the whole
likelihood in accordance with their respective prominence. For every likelihood in the recognition, the
Gaussians are reordered in a descending manner according to their weighted values in Eq. (3).
Denoting the values of reordered Gaussians Gi (i = 1, 2, …, 32) as Vj, we have V1 > V2 > V3 > … >V32 .
Then, the average contribution of a certain Vj can be evaluated in a statistical way as follows:
Bj 
Vj

 p
, j  1, 2, , 32
(6)

Here, p is the likelihood computed with Eq. (3), and Ω is the set of all likelihoods in the experiment.
The average contribution of the most prominent 20 Gaussians is shown in Table 1. Other Gaussians
are omitted here since their contribution is too small to be considered.
Fig. 4 Log Likelihoods Distribution of Gaussian Components
Table 1 Statistical Contribution of Different Gaussians in Accordance with Their Prominence
j
1
2
3
4
5
6
7
8
9
10
Bj
0.858943
0.103483
0.023988
0.007666
0.002991
0.001345
0.000668
0.000358
0.000205
0.000122
j
11
12
13
14
15
16
17
18
19
20
Bj
0.000077
0.000050
0.000033
0.000022
0.000016
0.000011
0.000008
0.000006
0.000004
0.000003
From Fig. 4 and Table 1, we can see firstly that not many Gaussians in the mixture make dominant
contribution to the likelihood, and the most prominent 10 Gaussians make statistically more than
99.97% contribution to the whole likelihood. Secondly, the contribution of the maximum Gaussian is
much more greater than others and the logarithm of the maximum mixture component is numerically
close to the log likelihood of the whole GMM. That is to say that only a few Gaussians, or just one
Gaussian in some cases, will dominate the likelihood of a GMM. So, the computation of a Gaussian
mixture model can be truncated to a small number of Gaussians or even one Gaussian provided that
the approximation accuracy is guaranteed. Since the computation is in log scale, the Gaussian
components with very small log likelihood values can be discarded in the whole likelihood
computation, without introducing significant loss of accuracy. This basic understanding led to the idea
of Gaussian selection [2, 5]. Many different algorithms have been proposed to decide which Gaussians
in the mixture dominate the likelihood [5-10]. The set of selected Gaussians is usually called the
shortlist of the mixture model.
The most commonly used Gaussian selection technique is the VQ-based Gaussian selection [5].
Though many different methods can be used to implement this technique, the key idea is to partition
the acoustic space into a number of sub-spaces, called clusters, each of which being represented by a
centroid. After training the HMM models, for each state-centroid pair (S, C) several dominant
Gaussians are selected according to a certain distortion measure between the centroid and the
Gaussians to form a shortlist for S. During recognition, each observation vector is mapped to a certain
centroid C first, and then the likelihood of each state S is computed only on the shortlist corresponding
to the (S, C) pair. Essentially, the VQ-based Gaussian selection technique is a two-pass search. In the
first pass, a rough model is used to determine the location of the observation vector in the acoustic
space, and a shortlist is correspondingly decided for each GMM. In the second pass, the likelihood of
each GMM for emitting the input vector is evaluated only on the derived shortlist and thus the
likelihoods of all GMMs are evaluated. Algorithmic steps of Gaussian selection can be summarized as
follows:
1) Classifying all the Gaussians of the GMMs in m clusters, in unsupervised manner;
2) Estimating the centroid of each cluster;
3) For each frame x, calculating and ordering the distances (probabilities) of x for each cluster;
4) Calculating the likelihood of each state limited to the shortlist of the most significant cluster or
to the shortlists of several most significant clusters.
The major differences between various Gaussian selection techniques reside in the way of clustering
the Gaussians and use different search mechanisms to map the input vector onto a centroid. The most
common one is the Gaussian selection based on vector-quantization (VQ) in the full dimensions of
mean vectors [5]. After system training, all mixture components are clustered into neighborhoods to
generate a set of codewords. Then, all the Gaussians of each state are assigned to a certain codeword,
i.e., a shortlist of Gaussians should be assigned to each codeword for each state. An important issue for
clustering and codebook generation is how to define the distance between two Gaussians. Various
distance measures, such as Euclidean distance, Mahalanobis distance and Kullback-Leibler divergence,
can be used here, thus leading to different VQ-based Gaussian selections [6, 8]. Other algorithms,
including subvector quantization (SVQ) [4], k-dimensional binary-tree searching [7] and Bucket Box
Intersection algorithm [3, 9], can also be used to determine the location of the input vector in the
acoustic space and therefore to determine the shortlists of significant Gaussians. It is reported that the
time saving of different Gaussian selection techniques in likelihood computation ranges from 27% to
80%, without considering the overhead of the first search pass. In using any Gaussian selection
technique, the major issue is the necessity to trade-off between rough model computation (i.e. the VQ
codebook searching) and accuracy degradation. Usually, a more detailed model (e.g., a higher-ordered
VQ codebook) gives better accuracy, but results in more computational overhead.
To implement the two-pass search procedure of Gaussian selection, the short lists for all state-centroid
pairs should be stored in memory as a core data structure of the ASR system. Though there is a
computational saving due to Gaussian selection, extra memory requirement is introduced because of
the use of the shortlists, implying an unignorable memory overhead. For example, in our experiments
with triphone acoustic models trained on WSJ0 corpus (the detail will be described in Section VI), the
number of tied states is 6000. The indices of Gaussians are programmed in the data type of short
integer. Assuming a 512-codeword VQ codebook is used for Gaussian selection and every shortlist
contains 4 Gaussians, it would require 24MB of memory space for shortlists storage. Such a memory
overhead is heavy in system implementation, especially for portable devices where the resource is
extremely constrained. Even though some programming skills could be used to pack several indices
into one byte in order to reduce memory consumption, accessing such indices would introduce
additional time overhead and therefore the real time performance would deteriorate further.
IV. PARTIAL DISTANCE ELIMINATION TECHNIQUE
A nearest-neighbor approximation method, which requires no additional memory for shortlists, can be
used as an alternative fast likelihood computation technique to reduce the computational overhead [12,
13]. Instead of computing the likelihood by summing across all Gaussians, the maximum Gaussian
probability is taken as the state likelihood. This nearest-neighbor approximation can be expressed as:

log p  x n S
 


1 N xnq   kq

max  logZ k   
2
1 k  M
2 q 1
 kq


 
2



(7)
Such a nearest-neighbor search problem can be thought as a vector quantization (VQ) codebook search
problem, where the Gaussians in that state are the codewords and the distortion measure is given on
right-side of Eq. (7). Let Dk(xn|y) denote the distortion measure for the codebook search (here, y is the
codebook), then

Dk  x n y   logZ k    xnq   kq
N
q 1

2
1
2
2 kq
(8)
The distortion measure is a log-likelihood, so we must maximize it in the search. By inspecting Eq. (8),
we can find that the right-hand side is actually a Mahalanobis distance measure, and the computation
of the distortion measure D(xn|y) is performed iteratively on each element of the observation vector.
Furthermore, with the progress of each recursion, the value of D(xn|y) decreases monotonically.
Therefore, a technique called partial distance elimination (PDE) [11] can be used to reduce the
computational complexity. The codeword which has maximum D(xn|y) can be decided without always
computing over all the range of feature vector elements; i.e., we can decide whether a codeword is a
possible candidate of the “best” one for the observation vector before completing the computation of
the codeword distortion over all elements of the observation vector. The algorithm is as follows.
Algorithm:
INPUT:
OUTPUT:
PROCEDURE
Partial Distance Elimination
xn , an N-dimensional observation vector;
 N  xn ; Z k , μk , Σ k  , k  1, 2, , M  , a GMM with M mixture components;
Dmax , the maximum logarithm of M mixture densities .
BEGIN
(1) Initialization
(1.1) Compute D1 according to Eq. (8) ;
(1.2) Dmax =: D1 ;
(1.3) k =: 2 ;
(2) WHILE (k <= M) DO
BEGIN
(2.1) Dk =: logZ k  ;
(2.2) q =: 1 ;
(2.3) WHILE (Dk >= Dmax) DO
BEGIN

(2.3.1) Dk =: Dk  xnq   kq

1
2
2
2 kq
;
(2.3.2) q =: q+1 ;
(2.3.3) IF (q > N)
(2.3.4.1) Dmax = Dk ;
(2.3.4.2) BREAK ;
ENDIF
END
(2.4) k =: k+1 ;
END
(3) RETURN Dmax ;
END
The algorithm starts by accumulating all the Mahalanobis distances and deriving the distortion
measure for the first Gaussian of the mixture, according to (8). The value of this distortion measure is
used to initialize a variable Dmax. For many other Gaussians in the mixture, Dk(xn|y) < Dmax . In
evaluating such a Gaussian, the intermediate value of the distortion will drop below Dmax at a certain
element j (j<N). So the recursion for evaluating such a Dk(xn|y) can be stopped at the jth element. This
means that the computation of a part of the Mahalanobis distances is eliminated in searching the
codebook. This kind of PDE algorithm can increase the efficiency of the codebook search, and
therefore speed up the likelihood approximation.
PDE technique may vary in the algorithmic efficiency. An extreme case is when the first Gaussian
happens to be the “best” one, so that for each of the remaining Gaussians a certain part of the
computation is eliminated in the searching procedure. This leads to the greatest saving of computation.
In the opposite case, the values of all the Gaussians on the observation vector are in the ascending
order. The distortion evaluation in the codebook search should be completely performed on every
element of the input vector for each Gaussian. That means no computation is eliminated in the
codebook search. Therefore, the efficiency of PDE technique relies heavily on how quickly a high
estimate of Dmax is obtained. A high efficiency can easily be accomplished by exploiting the high
correlation between adjacent observation vectors. Using the previous “best” Gaussian as the prediction
of the current “best”, and computing the distortion of this Gaussian first could result in a high Dmax
immediately and speed up the elimination process for the codebook search. This method is called the
“best mixture prediction” (BMP). We have performed a statistical analysis of the BMP approach on
TIMIT, the result is shown in Table 2. It can be seen that in 52.76% of the cases the BMP predicts the
best Gaussian for the current observation, while predicts the second best Gaussian with the probability
of 17.59%, etc. So, in most cases the BMP approach can predict a quite “good” Gaussian and can
efficiently reduce the needed computation for searching the best Gaussian.
Table 2 Statistical Analysis of BMP
Vj
BMP Probability (%)
Vj
BMP Probability (%)
V1
52.76
V6
2.32
V2
17.59
V7
1.67
V3
8.60
V8
1.27
V4
5.07
V9
1.00
V5
3.27
V10
0.79
The feature element reordering (FER) can complement PDE and BMP techniques for further reduction
in likelihood computation. The idea of FER is to shuffle the elements in the computation of Eq. (8) in
such a way that elements contributing prominently in the Mahalanobis distance are computed first
followed by the elements contributing less. With FER, PDE process is further speeded up because the
Gaussians whose probabilities are smaller than the current Dmax are eliminated as early as possible. In
FER, the reorder rule is usually learned offline from a portion of the development set and remains
fixed during recognition.
Though combining PDE framework with both BMP and FER is reported as an efficient technique to
compute the likelihood for LVSCR [12, 13], this kind of nearest-neighbor search techniques has a fatal
flaw. It uses only the probability of one Gaussian, though the greatest one, in the mixture to
approximate the whole likelihood. All other Gaussians are discarded and the contribution of these
Gaussians to the likelihood is omitted in the approximation. From Table 1, it is obvious that the
likelihood approximation error of this technique is statistically at about 15% — a value which is not
negligible. Thus, the resolution of the HMM model has been degraded, as well as the recognition
accuracy.
V. DYNAMIC GAUSSIAN SELECTION
To overcome the problem of the above nearest-neighbor approximation method, we propose an
alternative scheme called dynamic Gaussian selection (DGS). This scheme aims at utilizing the
advantages of PDE, BMP and FER techniques to speed up the computation of likelihood, while
obtaining a high recognition accuracy which is better than that of the nearest-neighbor approximation.
In DGS scheme, a shortlist of Gaussian components is selected to compute the likelihood instead of
using only the maximum Gaussian to approximate the likelihood. But the generation of the Gaussian
shortlist is totally different from the static shortlist generation in VQ-based Gaussian selection. DGS
scheme uses a dynamic data-driven method to generate the Gaussian shortlist for each observationstate pair. Unlike the two-pass search in VQ-based Gaussian selection, there is no pre-decided
Gaussian shortlist in DGS. DGS scheme does not map the observation vector to a certain centroid in
the acoustic space before the likelihood computation. The Gaussian shortlist is generated dynamically
during the computational procedure of likelihood, according to a heuristic knowledge about the
distance between each Gaussian and the best one to date. It is thus a single-pass search, within which
not only the Gaussian shortlist is decided but likelihood is computed as well. Another feature of DGS
scheme is that, since no shortlist is formed statically in advance of likelihood computation, no extra
memory is required for the storage of static Gaussian shortlists. The algorithm of this DGS scheme is
described below.
Algorithm:
INPUT:
OUTPUT:
PROCEDURE
BEGIN
Dynamic Gaussian Selection
xn , an N-dimensional observation vector ;
 N  xn ; Z k , μk , Σ k  , k  1, 2, , M  , a GMM with M mixture components ;
Qthresh , a threshold number of loops on right-hand side of Eq. (8) ;
Dapprox , the approximation of log likelihood of the GMM .
(1) Compute DBMP , the log likelihood of the BMP Gaussian component;
(2) Dmax =: DBMP ;
(3) Dapprox =: Dmax ;
(4) WHILE (the algorithm has not traversed all Gaussians) DO
BEGIN
(4.1) Perform PDE on an untouched Gaussian N  xn ; Z k , μk , Σ k  ;
(4.2) IF (after Qthresh loops the intermediate value of Dk is not less than Dmax )
(4.2.1) Complete the loops to derive Dk of the Gaussian ;
(4.2.2) Dapprox =: logadd [ Dapprox + Dk ] ;
(4.2.3) IF (Dk > Dmax)
(4.2.3.1) Dmax =: Dk ;
ENDIF
ENDIF
END
(5) RETURN Dapprox ;
END
The algorithm of DGS is based on the integration of PDE, BMP and FER; that means that it is an
extension of PDE framework combined with both BMP and FER. The basic idea of this algorithm is to
use the number of loops at which the recursion on right-hand side of Eq. (8) stops as a clue to decide
whether the Gaussian should be included in the shortlist. In this algorithm, the BMP Gaussian is
computed first and its value is assigned to Dmax. Then, each Gaussian is evaluated using the standard
PDE algorithm. For a Gaussian, as described earlier, with PDE framework, the summation loop of the
right-hand side of Eq. (8) will stop at the jth element (j<N) if the intermediately accumulated Dk drops
below the current Dmax . The stopping index j can be considered as a heuristic hint of the distance
between the current Dmax and the likelihood of the computed Gaussian. If j is a number of small value,
i.e., the summation loops stop at an early element, then the log likelihood of this Gaussian component
is far more less than the BMP Gaussian since the value of Eq. (8) decreases monotonically with the
progress of each loop. Therefore, the smaller the value of j, the greater the distance between this
Gaussian component and the BMP Gaussian will be. This means that Gaussian components with a
small value of j contribute little in the whole likelihood of the state and thus can be omitted in the
likelihood computation. Otherwise, if j is a large number, the summation loop of Eq. (8) stops at a later
element. This indicates that the log likelihood of this Gaussian component is close to the current BMP
Gaussian because the elements of the Gaussian component are reordered in such a way that elements
with higher contribution to the distortion measure in Eq. (8) are computed first, followed by the
elements contributing less. In other words, the contribution of the Gaussian component to the whole
likelihood is comparable to that of the current Dmax and thus it should be included in the shortlist. In
the algorithm, a threshold number Qthresh is given to decide whether a Gaussian component should join
the shortlist. If j is greater than Qthresh , the Gaussian component is selected to be included in the
shortlist and all the loops of Eq. (8) for this Gaussian component are completed in order to include its
full contribution in the likelihood. All the selected Gaussian components constitute the shortlist and the
likelihood of the state is computed restrictively on this shortlist. The shortlist is decided dynamically in
the procedure of likelihood computation. PDE technique is used here to reduce the computational
complexity. But instead of using only the maximum Gaussian component to approximate the
likelihood, a shortlist of Gaussians is used. So, DGS scheme is an extended PDE technique in terms of
that a Gaussian shortlist is decided based on PDE framework. Of course, the length of the dynamical
shortlist is decided by the value of Qthresh . With a small Qthresh , the dynamical shortlist is relatively
long and thus the computational saving is low, though the degradation of accuracy is relatively small.
The value of Qthresh can be adjusted in the implementation of DGS to make a compromise between the
computational cost and the recognition accuracy. In comparison with VQ-based Gaussian selection
methods in which static shortlists should be stored for all state-centroid pairs, DGS scheme is memory
saving because no shortlist should be pre-decided and kept in memory.
VI. EXPERIMENTS AND RESULTS
A. System Descriptions
Experiments on continuous speech recognition tasks have been carried out to evaluate and compare the
performance of DGS scheme with that of PDE. In the experiments, large vocabulary speech
recognition toolkit HTK 3.4 is used as the baseline system. The likelihood computation modules in
HTK 3.4, such as the function cSOutP( ) in the file HRec.c and the function IDOutP( ) in the file
HModel.c, have been modified to implement both PDE and DGS schemes. Two accent-variant large
vocabulary continuous speech corpora of English, TIMIT and WSJ0, are used to perform the
recognition.
For the recognition experiment on TIMIT, 8 sentences from each of the 462 speakers in the corpus are
used for training; i.e., the training set consists of 3696 recordings. The testing set consists of 8
sentences from each of 168 speakers other than the speakers in the training set. CMU phoneme set is
used and 40 continuous HMMs for monophones are used as the acoustic model, including an HMM
for silence. All HMMs have 3-state, left-to-right topology with the same number of Gaussian mixtures
ranging from 16 to 128. Phonetic recognition is performed, for which the language model is based on a
simple “word-loop grammar”.
To perform word-level recognition experiments on WSJ0, we follow the method and steps in the HTK
training recipe suggested in [32]. SI-84 and TIMIT training corpora are used to build cross-word
triphone models. The state transition matrices of triphones sharing the same central phone are tied. A
decision tree process using a list of linguistically motivated questions about a triphone’s context is
used to cluster and tie triphone states together. The parameters of the tied-state triphones are reestimated by using Baum-Welch training. The number of Gaussians per-state ranges from 8 to 64 in
the resulting triphone models while the number of tied-states is 6000. To evaluate and compare the
performance of different likelihood computation algorithms, recognition experiments are performed on
the Nov.’92 ARPA WSJ test set which contains 330 sentences from 8 speakers. The WSJ 5K nonverbalized closed vocabulary set and the WSJ standard 5K non-verbalized closed bigram language
model are used.
All speech data is coded into 12 MFCCs, C1 to C12, along with normalized log-energy C0 and the first
and the second time derivatives of these 13 coefficients, resulting in 39-dimensional feature vectors.
To complement PDE with FER, the 39 elements of the feature vector are shuffled according to their
contributions to the whole likelihood. The reordering is learned off-line from all the SA sentences (the
dialect sentences) in the TIMIT test set. The resulting element reordering is listed in Table 3.
Table 3 Element Reordering of 39-element MFCC Feature Vectors
Index
1
2
3
4
5
6
7
8
9
10
11
12
13
MFCC
Coefficient
C0
C1
∆C0
∆C1
∆∆C0
∆∆C1
C2
C3
C4
C5
∆C2
∆∆C2
C6
Index
14
15
16
17
18
19
20
21
22
23
24
25
26
MFCC
Coefficient
C7
∆C4
∆∆C4
∆C3
∆C5
C8
∆∆C3
C9
∆∆C5
C11
∆C6
∆∆C6
∆C7
Index
27
28
29
30
31
32
33
34
35
36
37
38
39
MFCC
Coefficient
∆∆C7
∆∆C8
∆C8
∆∆C9
∆C9
C10
C12
∆∆C10
∆∆C11
∆C11
∆C10
∆∆C12
∆C12
B. Results
The performance of PDE and DGS schemes are assessed by carrying out three experiments and
calculating the total recognition time, time for likelihood computation in the recognition tasks, as well
as the recognition accuracy. In the implementation of DGS scheme, the threshold number Qthresh is set
to be 35 according to an empirical study. All experiments are performed on a 3.4GHz Intel Pentium 4
machine with 2GB RAM. The first experiment is a phone-level recognition on TIMIT in which the
phone-level accuracy is calculated instead of word or sentence accuracy in order to clearly
demonstrate the differences between performances of different schemes. Fig. 5 and Table 4 show the
results of this phone-level recognition experiment on TIMIT. Different HMMs with different number
of Gaussian mixtures are trained and applied in the recognition. The number of Gaussian mixtures
ranges from 16 to 128. In Fig. 5, every recognition time is normalized by the recognition time of the
corresponding baseline. The results of the second experiment, which is a word-level recognition
experiment on WSJ0, are shown in Fig. 6 and Table 5. In this experiment, the number of Gaussian
mixtures in the triphone models ranges from 8 to 64. Also in Fig. 6, every likelihood computation time
is normalized by its counterpart of the corresponding baseline. The third experiment aims at evaluating
and comparing the real-time performance and accuracy of PDE and DGS schemes at different pruning
thresholds for Viterbi beam search. In this experiment, a series of word-level recognition has been
performed on WSJ0 with the same set of 32-Gaussian acoustic models. The pruning threshold of the
general-purpose Viterbi word recognizer HVite is tuned from 250 to 180 in a step of 10. The results of
this experiment are presented in Table 6 and Table 7. It’s worth mentioning that every result shown
here is an average of 10 runs on each configuration in order to reduce interference from outside
processes.
1.00
Baseline
PDE
PDE+BMP+FER
DGS
0.90
0.80
0.70
0.60
16-G
32-G
64-G
128-G
Fig. 5 Normalized Total Recognition Time of PDE, PDE+BMP+FER, and DGS on TIMIT Corpus
Table 4 Phone Recognition Accuracy (%) of PDE, PDE+BMP+FER, and DGS on TIMIT Corpus
Number of Gaussian Mixtures
16-G
32-G
64-G
128-G
56.68
58.69
59.88
60.18
56.28
58.27
59.43
59.70
56.28
58.27
59.43
59.70
56.63
58.70
59.83
60.13
Scheme
Baseline
PDE
PDE+BMP+FER
DGS
1.00
Baseline
PDE
PDE+BMP+FER
DGS
0.90
0.80
0.70
0.60
8-G
16-G
32-G
64-G
Fig. 6 Normalized Likelihood Computation Time of PDE, PDE+BMP+FER, and DGS on WSJ0
Corpus of 330 Sentences
Table 5 Word Recognition Accuracy (%) of PDE, PDE+BMP+FER, and DGS on WSJ0 Corpus of
330 Sentences
Scheme
Baseline
PDE
PDE+BMP+FER
DGS
Number of Gaussian Mixtures
8-G
16-G
32-G
64-G
92.90
93.24
93.37
91.71
92.90
93.15
93.29
91.74
92.90
93.15
93.29
91.74
92.90
93.26
93.33
91.78
Table 6 Sentence Correctness (%) of PDE+BMP+FER and DGS on WSJ0 Corpus of 330 Sentences at
Different Beam Thresholds
Scheme
Baseline
PDE+BMP+FER
DGS
250
50.61
49.70
50.61
240
49.70
48.79
49.70
0
230
47.27
46.36
47.27
Beam Threshold
220
210
45.76
42.42
45.15
41.52
45.76
42.73
200
37.88
37.58
37.88
190
35.87
34.85
35.87
180
30.30
30.00
30.30
Table 7 Real-time Factor (CPU Time / Audio Time) of PDE+BMP+FER and DGS on WSJ0 Corpus
of 330 Sentences at Different Beam Thresholds
Scheme
Baseline
PDE+BMP+FER
DGS
250
3.46
3.31
3.30
240
2.92
2.77
2.73
230
2.44
2.30
2.29
Beam Threshold
220
210
1.65
2.02
1.89
1.53
1.88
1.53
200
1.33
1.22
1.22
190
1.07
0.98
0.99
180
0.87
0.77
0.78
The experimental results of the first experiment indicate that DGS scheme achieves a significant time
saving (>23%) in phone-level recognition with such a small degradation of accuracy that it can be
negligible. To describe the performance of DGS algorithm and to make a comparison with the
performance of PDE schemes, we use the 128-Gaussian case for the experiment on TIMIT as an
example. PDE scheme speeds up the likelihood computation by 16.00%, while the recognition
accuracy is degraded from 60.18% to 59.70%. Incorporating PDE scheme with both BMP and FER
further saves the likelihood computation time by 9.24%, without introducing any more degradation of
phone recognition accuracy because both BMP and FER do not change the basic mechanism of PDE
to choose the “best” Gaussian to represent the mixture. DGS scheme saves the likelihood computation
time by 25.44%, whereas the degradation of the recognition accuracy is only 0.05%. Confidence
interval measuring of the results indicates that there is almost no difference between the baseline and
DGS in terms of accuracy while a slight difference exists between PDE and baseline. For the wordlevel recognition in the second experiment, DGS scheme also performs better than PDE scheme in
both time overhead and recognition accuracy, though the differences in accuracy between different
schemes are not significant since grammatical knowledge has been introduced in the word-level
recognition. It should be pointed out that the word accuracies of different schemes for 64-Gaussian
models are lower than the word accuracies for 32-Gaussian models. In addition, the word accuracies of
PDE and DGS for 64-Gaussian models are both better than that of the baseline. A plausible
explanation is that the model complexity is too high thus the HMMs can not be well trained.
It is noticeable from Fig. 5 and Fig. 6 that DGS algorithm takes a little more time to compute the
likelihood than the scheme of PDE incorporated with both BMP and FER. The extra time cost comes
from the completion of the summation loop of Eq. (8) for computing the Gaussians in the dynamic
shortlists and the logadd[] computation of these Gaussians. This extra time overhead depends on the
number of Gaussians in the dynamic shortlists, i.e., on the lengths of the dynamic shortlists. We have
inspected the average lengths of the shortlists in DGS algorithm with Qthresh given the value of 35, in
the cases of different number of Gaussian mixtures. The results are illustrated in Table 8. In Table 4
and 5, it is shown that when Qthresh is set to 35 the recognition accuracy of DGS algorithm is almost the
same as that of the baseline. Table 8 indicates that this high recognition accuracy is achieved with the
average length of the dynamic shortlists less than three. That means, in each dynamic shortlist, mixing
two “sub-optimal” Gaussians on average with the “best” Gaussian is enough to achieve almost the
same recognition accuracy as the baseline. This result coincides with the statistics in Table 1 which
shows that the three most significant Gaussians contribute more than 98.64% on average to the whole
likelihood of a GMM model. DGS algorithm merely needs to complete the summation loop of (8) for
two more Gaussians on average than PDE+BMP+FER does. Therefore the extra time cost of DGS
algorithm is quite limited since the average length of the dynamic shortlists is small.
Table 8 Average Length of Dynamic Shortlists with Qthresh = 35
Number of Gaussian Mixtures
16
32
64
128
Average Length of Dynamic Shortlists
2.82
2.10
2.43
2.80
From the results in Table 6 and Table 7,we can see that at every different beam threshold, the sentence
correctness of DGS scheme is nearly always the same as that of the baseline, while the sentence
correctness of PDE+BMP+FER scheme is noticeably inferior. So, to derive the same recognition
correctness of PDE+BMP+FER scheme with a certain beam, we can employ a narrower beam in DGS
scheme in order to reduce the time overhead for beam search. In addition, at a certain beam threshold
which is large enough (240, for instance), DGS scheme performs better than PDE+BMP+FER scheme
does in terms of the overall recognition time, even though DGS takes more time to compute the
likelihoods in the recognition procedure. The reason why the real-time factor of DGS scheme is better
than that of PDE+BMP+FER at a large beam threshold is that the PDE scheme takes only the
maximum Gaussian probability to approximate the state likelihood. So, the resolution of the GMM
models is reduced by this nearest-neighbor approximation method. With PDE scheme, the likelihoods
of many triphone states are the same, therefore the beam search incurs high cost of computation due to
the narrow range of likelihood values. However, DGS scheme uses a shortlist of most prominent
Gaussians to approximate the GMM model, so the precision of GMM models is retained. By
examining all cases in the experimental results, it can be concluded that DGS scheme is an effective
way to speed up likelihood computation in continuous speech recognition with almost no degradation
in accuracy.
VII. CONCLUSION
This paper has proposed a fast likelihood computation technique, called Dynamic Gaussian Selection
(DGS), based on the concept of Gaussian selection. Different from the VQ-based Gaussian selection
techniques, this approach is a one-pass search technique which generates a dynamic shortlist of
Gaussians for each state during the procedure of likelihood computation. DGS algorithm is an
extension of both techniques of PDE and BMP. It uses the number of summation loops in the Gaussian
computation to dynamically decide a small set of “sub-optimal” Gaussians which are numerically
close to the “best” Gaussian. These dynamically selected Gaussians are combined with the “best”
Gaussian to constitute the Gaussian shortlist. Though the theoretic gain of DGS remains to be analyzed,
experiments show that computing likelihoods restrictively on such dynamic shortlists can significantly
speed up the likelihood computation while introducing almost no additional recognition error. It can be
used as an alternative technique to perform fast likelihood computation. Compared to the VQ-based
Gaussian selection techniques, the advantage of DGS is that it does not require extra memory for the
storage of Gaussian shortlists. This prominent feature makes it particularly suitable for applications on
embedded platform to save the consumption of the memory.
Though DGS algorithm is an effective technique to achieve highly accurate likelihood computation, it
can only speed up the likelihood computation to a limited extent. That is to say, if DGS is used solely
as an alternative to perform likelihood computation in a speech recognition system, we could not
derive satisfactory improvement of real-time performance. To improve the speed of likelihood
computation as much as possible, we should integrate DGS with other optimization techniques, such
as model optimization of GMMs and parallel computation. One promising way to do this integration is
to apply the context-independent HMM-based two-pass search used in Julius system to perform
decoding while using DGS to compute each likelihood and programming the algorithm with SIMD
technology.
REFERENCES
[1] X. Huang, A. Acero, and H.-W. Hon, “Spoken Language Processing: A Guide to Theory
Algorithm and System Development,” Prentice Hall PTR, New Jersey, April 2001.
[2] M. J. F. Gales, K. M. Knill, and S. J. Young, “State-Based Gaussian Selection in Large Vocabulary
Continuous Speech Recognition Using HMM’s,” IEEE Trans. on Speech and Audio Processing, Vol. 7,
No. 2, pp. 152-161, March 1999.
[3] S. Srivastava, “Fast Gaussian Evaluations in Large Vocabulary Continuous Speech Recognition,”
Master’s Degree Thesis, Department of Electrical and Computer Engineering, Mississippi State
University, December 2002.
[4] M. Ravishankar, R. Singh, B. Raj, and R. M. Stern, “The 1999 CMU 10X Real Time Broadcast
News Transcription System,” in Proc. of Speech Transcription Workshop, Maryland, 2000.
[5] E. Bocchieri, “Vector Quantization for the Efficient Computation of Continuous Density
Likelihood,” in Proc. of ICASSP’93, Vol. 2, pp. 692-695, April 1993.
[6] K. M. Knill, M. J. F. Gales, and S. J. Young, “Use of Gaussian selection in large vocabulary
continuous speech recognition using HMMS,” in Proceedings of the Fourth International Conference
on Spoken Language, Oct. 1996, Vol. 1, pp. 470—473.
[7] S. Ortmanns, H. Ney, and T. Firzlaff, “Fast Likelihood Computation Methods For Continuous
Mixture Densities In Large Vocabulary Speech Recognition,” in Proceedings of Eurospeech’97, pp.
139—142.
[8] G. Linares, P. Nocera, and D. Matrouf, “Partitionnement Dynamique des Distributions pour le
Calcul des Emissions dans un Décodeur Acoustico-Phonétique Markovien,” XXIIIèmes Journées
d’Etude sur la Parole, Aussois, pp. 177-180, Juin 2000.
[9] J. Fritsch, and I. Rogina, “The Bucket Box Intersection (BBI) Algorithm For Fast Approximation
Evaluation of Diagonal Mixture Gaussians,” Proc. of ICASSP’96, Vol. 2, pp. 837-840, May 1996.
[10] T. Kinnunen, E. Karpov, and P. Fränti, “Real-Time Speaker Identification and Verification. IEEE
Trans. on Audio, Speech, and Language Processing, Vol. 14, Issue 1, pp. 277- 288, Jan. 2006.
[11] Chang-Da Bei, and R. Gray, “An Improvement of the Minimum Distortion Encoding Algorithm
for Vector Quantization,” IEEE Transactions on Communications, Vol. 33, No. 10, Oct. 1985, pp.
1132—1133.
[12] B. L. Pellom, R. Sarikaya, and J. H. L. Hansen, “Fast Likelihood Computation Techniques in
Nearest-Neighbor Based Search for Continuous Speech Recognition,” IEEE Signal Processing Letters,
Vol. 8, No. 8, pp. 221-224, August 2001.
[13] R. Sarikaya, “Robust and Efficient Techniques for Speech Recognition in Noise,” Ph. D.
Dissertation, Department of Electrical and Computer Engineering, Duke University, March 2001.
[14] M. Hwang and X. Huang, “Acoustic Distribution Clustering in Phonetic Hidden Markov
Models,” in Proceedings of Eurospeech ’91, pp. 785—788.
[15] S. J. Young and P. C. Woodland, “The Use of State Tying in Continuous Speech Recognition,” in
Proceedings of Eurospeech ’93, Berlin, 1993, pp. 2203—2206.
[16] M. Hwang and X. Huang, “Subphonetic modeling for speech recognition,” in Proceedings of the
Workshop on Speech and Natural Language, Harriman, New York, 1992 pp. 174—179.
[17] M. Hwang and X. Huang, “Shared-distribution hidden Markov models for speech recognition,”
IEEE Transactions on Speech and Audio Processing, Vol. 1, No. 4, 1993, pp. 414—420.
[18] X. Huang, K. F. Lee, and H. W. Hon, “On semi-continuous hidden Markov modeling,” in
Proceedings of ICASSP ’90, Vol.2, pp. 689—692.
[19] V. V. Digalakis, P. Monaco, and H. Murveit, “Genones: Generalized Mixture Tying in
Continuous Hidden Markov Model-Based Speech Recognizers,” IEEE Transactions on Speech and
Audio Processing, Vol. 4, No. 4, 1996, pp. 281—289.
[20] G. F. G. Yared, F. Violaro, and L. C. Sousa, “Gaussian Elimination Algorithm for HMM
Complexity Reduction in Continuous Speech Recognition Systems,” in Proceedings of
INTERSPEECH, 2005, pp. 377—380.
[21] E. Bocchieri and B. K. Mak, “Subspace Distribution Clustering Hidden Markov Model,” IEEE
Transactions on Speech and Audio Processing, Vol. 9, No. 3, 2001, pp. 264—275.
[22] M. Ravishankar, R. Bisiani, and E. Thayer, “Sub-Vector Clustering to Improve Memory and
Speed Performance of Acoustic Likelihood Computation,” in Proceedings of Eurospeech ’97, pp. 151
—154.
[23] “MPI Documents,” http://www.mpi-forum.org/docs/
[24] S. Kanthac, K. Schutz, and H. Ney, “Using SIMD instructions for fast likelihood calculation in
LVCSR,” in Proceedings of ICASSP ’00, Vol. 3, June, 2000, pp. 1531—1534.
[25] M. Saraclar, M. Riley, and E. Bocchieri, et al, “Towards Automatic Close Captioning: Low
Latency Real-time Broadcast News Transcription”, in Proceedings of the 7th International Conference
on Spoken Language Processing (ICSLP-2002), Sept. 2002, pp. 1741—1744.
[26] J. L. Ou, J. Cai, Q. Lin, “Using SIMD Technology to Speed up Likelihood Computation in
HMM-based Speech Recognition Systems,” (Accepted by 2008 International Conference on Audio,
Language and Image Processing (ICALIP 2008), July 7-9, 2008, Shanghai).
[27] A. Chan, J. Sherwani, M. Ravishankar, and A. Rudnicky, “Four-Layer Categorization Scheme of
Fast GMM Computation Techniques in Large Vocabulary Continuous Speech Recognition Systems,”
in Proceedings of INTERSPEECH, 2004, pp. 689—692.
[28] A. Lee, T. Kawahara, and K. Shikano, “Gaussian Mixture Selection Using Context-Independent
HMM,” in Proceedings of ICASSP ’01, May 2001, Vol. 1, pp. 69—72.
[29] A. Lee, T. Kawahara, and K. Shikano, “Julius — An Open Source Real-time Large Vocabulary
Recognition Engine,” In Proc. of EUROSPEECH 2001, Sept. 2001, pp. 1691—1694.
[30] X. Li, and J. Bilmes, “Feature Pruning in Likelihood Evaluation of HMM-Based Speech
Recognition,” in Proceedings of ASRU 2003, pp. 303—308.
[31] X. Li, and J. Bilmes, “Feature Pruning for Low-Power ASR Systems in Clean and Noisy
Environments,” IEEE Signal Processing Letters, Vol. 12, No. 7, July 2005, pp. 489—492.
[32] K. Vertanen, “Baseline WSJ Acoustic Models for HTK and Sphinx: Training Recipes and
Recognition Experiments,” Technical Report, Cavendish Laboratory, 2006.