DEVELOPMENT OF A NON-ITERATIVE METHOD FOR POSITIVE MATRIX
FACTORIZATION IN IMAGE COMPRESSION
2.1 Introduction
Data compression, ideally with no error or with minimum error, is desirable to
save storage requirements and time for data transmission. The above requirements are
more significant in the case of image data. Any colour image can be split into three
primary colour( Red, Green and Blue) images. The pixels (picture elements) of any such
primary colour image have intensities of the colour varying from minimum value (0%
saturation) to maximum value (100 % saturation). These pixel intensity (PI) values are
obviously positive. The 2D image data is normally represented in the form of a matrix
whose elements are the PI values. In general , a m x n pixels image is represented by a
m x n matrix. For example, a 256 x 256 pixels image is represented by 256 x 256
matrix of PI values. For convenience of processing, the image is normally subdivided
into smaller blocks of, usually, 8x8 pixels. In this research, block sizes of 2x2,4x4,8x8
and 16x16 pixels have been used for performance analysis. An optimum block size of
4x4 pixels has been arrived at.
A number of image compression methods available in the literature have been
reported in the previous chapter. Some of them are based on positive matrix
factorization using outer product expansion [25] and tensor decomposition [30-32].
Using positive matrix factorization method, the mxn PI values of a mxn image matrix
can be represented as the outer products of a mx1 matrix of m row factors and 1xn
matrix of n column factors. Thus a total of m+n factors are used to represent mxn PI
values, resulting in a compression factor of (mxn) / (m+n).
In this positive matrix factorization method, the computation of row factors and
column factors is based on iterative calculations. For example, all the row factors and
column factors are assumed to have an initial value of 1. Then, in a step by step
manner and in a cyclic order the row factors and column factors are incremented. At
each step the outer products are calculated, compared with the PI values and RMSE
(Root Mean Square Error) is calculated. This iterative process is repeated until the
RMSE reduces and converges to a minimum value which is the LRMSE (Least Root
Mean Square Error). For example, in the case of a 4x4 matrix, a total of 8 factors (4 row
factors and 4 column factors) are to be estimated. During iteration, the incremental
change in any one factor will usually result in need for further incremental changes in all
the other factors. Hence the processing time for iterations is large. A computationally
efficient non-iterative method of positive matrix factorization, which can be applied for
image compression, has been developed as a part of this research work and explained
next.
2.2. Non-iterative method of positive matrix factorization
Considering the example of a general 4x4 positive matrix I, it is to be factorized
in to a 4x1 matrix (  ) of row factors and a 1x4 matrix (C) of column factors. The outer
products of  (r1, r2, r3, r4) and C (c1, c2, c3, c4) are expected to ideally match the matrix
I. However in practice the outer products yield the matrix I .
 I11

I
I   21
 I 31

 I 41
 r1c1

r 2c1
I 
 r 3c1

 r 4c1
r1c 2
r1c 3
r 2c 2 r 2c 3
r 3c 2 r 3c3
r 4c 2 r 4c 3
 r1 
 
r2
 
 r3 
 
 r4 
r1c 4 

r 2c 4 
r 3c 4 

r 4c 4 
I12
I13
I 22
I 23
I 32
I 33
I 42
I 43
 I11

 I 21
 
 I 31
I
 41
C   c1 c2 c3 c4 
I14 

I 24 
I 34 

I 44 
I12
I13
I 22
I 23
I 32
I 33
I 42
I 43
C
I14 

I 24 

I 34 
I 44 
I
2.2.1 General procedure for factorization with minimum LMSE
Step 1
Let us consider a 4X4 block of an image with PIs I11, I12, ….. I44 as shown in
Figure 2.1. The sums SR1, SR2, SR3 , and SR4 of the 4 rows of the PIs of the block and
the sums SC1, SC2, SC3 and SC4 of the PIs of the 4 columns of the block are calculated
and indicated in Figure 2.1. The 16 PIs are to be factorized as the outer products of 4
row factors (r1, r2,r3,r4) and 4 column factors (c1,c2,c3,c4). Therefore, ideally r1c1= I11; r1c2
= I12…… r4c4= I44.However, in practice the factorization may not be exact and hence
r1c1=Ī11, r1c2 = Ī12 . . . . r4c4 = Ī44. The error is the sum of the difference between I values
and I values. The aim is to have Least Mean Square Error (LMSE).
c1
c2
c3
c4
r1 I11
I12
I13
I14
SR1
r2 I21
I22
I23
I24
SR2
r3 I31
I32
I33
I34
SR3
r4 I41
I42
I43
I44
SR4
Row sum
Row factors
Column factors
SC1 SC2 SC3 SC4
Column Sum
Figure 2.1: Block of 4x4 pixels represented by matrix I
Step 2
The estimated row factors (r1, r2,r3,r4) and the estimated
column factors
(c1,c2,c3,c4) result in r1c1= Ī11 , r1c2 = Ī12 . . . . r4c4 = Ī44 . Hence sum of the squared errors
of the 16 PIs will be equal to [ (I11 - Ī11)2 + (I12 - Ī12)2 + . . . . + (I44 – Ī44)2]. Considering the
logic that the larger values of PIs are likely to contribute large errors, the largest value
among the SR1, SR2, SR3 , SR4 ,SC1, SC2, SC3 and SC4 is identified. For example, let SR1
be the largest sum. To have no error in this row 1, it is chosen that c 1= I11, c2 = I12, c3 =
I13 , c4 = I14 and hence r1 =1. The remaining row factors r2, r3 and r4 are estimated in
such a way as to give minimum sum of squared errors in their respective rows 2, 3 and
4.
Step 3
The method of estimating r2 is given below and the same procedure is adopted to
estimate r3 and r4.
In row 2, error2 = E2 = [ (I21 – Ī21)2 + (I22 - Ī22)2 + (I23 – Ī23)2 + (I24 – Ī24)2 ]
= [(I21 – r2c1)2 + (I22 – r2c2)2 + (I23 – r2c3)2 + (I24 – r2c4)2 ]
= [(I21 – r2 I11)2 + (I22 – r2 I12)2 +
(I23 – r2 I13)2 + (I24 – r2 I14)2]
(2.2.1.1)
Differentiating E2 with respect to r2 and equating it to zero yields
2 [(I21 – r2 I11) (- I11)+ (I22 – r2 I12) (-I12)+
(I23 – r2 I13)(- I13) +(I24 – r2 I14) (- I14)] = 0
(2..2.1.2)
Solving this equation,
r2 = [ I11 I21 + I12 I22+ I13 I23 + I14 I24 ] / [ (I11)2 + (I12)2 + (I13)2 + (I14)2]
(2.2.1.3)
Similarly
r3 = [ I11 I31 + I12 I32+ I13 I33 + I14 I34 ] / [ (I11)2 + (I12)2 + (I13)2 + (I14)2]
(2.2.1.4)
r4 =[ I11 I41 + I12 I42+ I13 I43 + I14 I44 ] / [ (I11)2 + (I12)2 + (I13)2 + (I14)2]
(2.2.1.5)
For image compression, instead of transmitting the 16 PIs, the 4 values of row
factors (r1,r2,r3,r4) and the 4 values of column factors (c1,c2,c3,c4) alone are transmitted,
through an ideal error-free noiseless channel and received by the receiver . Using these
4 row factors and 4 column factors, the receiver is able to estimate the 16 PIs ( Ī11 , Ī12 . .
. Ī44 ), which are the outer products of row factors and the column factors . Of these 16
values, the four values corresponding to row 1 will be with no error, since Ī 11 = I11 ; Ī12 =
I12 ; Ī13 = I13 ; Ī14 = I14 .The remaining 12 PIs alone will contribute to the error. The
estimation of error is possible only at the transmitting end, which has the 16 original PIs
(I11, I12, ….. I44 ) and the 16 estimated PIs (Ī11 , Ī12 . . . Ī44 ).
The compression factor is 2, since 16PIs are recreated from a total of 8 values of
factors.
2.3 Numerical example for positive matrix factorization
A 4x4 block of the Cameraman image [55] is considered for explaining the process of
4x4 positive matrix factorization.
Step 1
The 16 PI values of 4X4 block are shown in matrix form in Figure.2.2. The sum of
the PIs in each row and column are shown along the respective rows and columns . The
largest among these eight sum values is identified as 632, corresponding to the first row
Column factors
c2
c3
c4
r1
156 157
160
159
632
r2
156
157
159
158
630
r3
158
157
156
156
627
r4
160
57
154
154
625
630 628
629
627
Row sum
Row factors
c1
Column Sum
Figure 2.2 : 16 PIs of a 4X4 pixels block shown in matrix form
Step 2
The sixteen PIs of the block are to be factorized as the outer products of 4 row
factors (r1, r2,r3,r4) and 4 column factors (c1,c2,c3,c4). Normally, after factorization the PIs
of 1st row which has the maximum sum is likely to contribute maximum error. To avoid
this error, the PIs of row 1 are chosen as the column factors (c1,c2,c3,c4) and hence r1 is
1, resulting in zero error from the 4 PIs of this row as shown in Figure 2.3.
156
157
160
159
1
156
157
160
159
r2
156
157
159
158
r3
158
157
156
156
r4
160
157
154
154
Figure 2.3: Choosing the column factor values
Step 3
The unknown row factors r2,r3 and r4 are estimated in such a way that
the
estimated product PI values of corresponding rows 2, 3, and 4 result in minimum error
in their respective columns. Using equations (2.2.1.3),(2.2.1.4) and (2.2.1.5),
r2 
(156 156)  (157 157)  (160 159)  (159 158) 99547

 0.9968
(1562  157 2  1602  1582 )
99866
r3 
(156 158)  (157 157)  (160 156)  (159 156) 99061

 0.9921
(1562  157 2  1602  1592 )
99866
r4 
(156 160)  (157 157)  (160 154)  (159 154) 98735

 0.9889
(1562  157 2  1602  1592 )
99866
Thus the values of r2 ,r3 and r4 are calculated to be 0.9968, 0.9921 and 0.9889
respectively giving a minimum of the squared error.
The column factors c1(156), c2(157), c3(160) and c4(159), along with the row
factors of r1(1), r2(0.9968), r3(0.9921) and r4(0.9889) are transmitted to the receiver. The
receiver will calculate the 16 outer products of the 4 row factors and 4 column factors.
Step 4
The 4X4 matrix can be reconstructed by the outer product of row factors and
column factors. The reconstruction for the matrix shown in Figure. 2.2 is shown below in
Figure .2.4 .
1
.9968
156
157
160
159
156
157
160
159
(156)
(157)
(160)
(159)
157
159
158
(157)
(159)
(158)
156
159
158
(157)
(156)
(156)
155
158
157
(157)
(154)
(154)
156
(156)
0.9921
155
(158)
0.9889
154
(160)
Figure 2.4: Reconstructed PIs along with original values in bracket for
comparison
The above 16 values are used to reconstruct the 4x4 pixels block.
Step 5
Making use of the original 16 PIs, and the reconstructed 16 PIs , the LRMSE
Least Root Mean Square Error) is calculated.
(
LRMSE = { [ (156 -156)2 +( 157-157)2 + (160-160)2 +(159-159)2 + (156-156)2 + (157157)2 + (159-159)2 + (158-158)2 + (158-155)2 + (157-156)2 + (156-159)2 + (156-158)2 +
(160-154)2 + (157-515)2 + (154-158)2 + (154-157)2 ]
/ 16} ½ = 2
2.4. Simulation results for image compression based on matrix
factorization
The proposed method of image compression based on factorization, as
explained in the previous sections, is applied to a set of test images [55] and the
simulated results are shown below.
(i)
(i)
(ii)
(iii)
(ii)
(iii)
(i)
(ii)
(iii)
Figure 2.5: Image compression based on matrix factorization of medical images. (i)
Original image, (ii).Reconstructed image based on Non-iterative matrix factorization, (iii)
Reconstructed image based on iterative matrix factorization
(i)
(ii)
(iii)
(i)
(ii)
(iii)
(i)
(ii)
(iii)
Figure 2.6: Image compression based on matrix factorization of Electron-microscopic
images. (i) Original image, (ii).Reconstructed image based on Non-iterative matrix
factorization, (iii) Reconstructed image based on iterative matrix factorization
(i)
(ii)
(iii)
(i)
(i)
(ii)
(ii)
(iii)
(iii)
Figure 2.7: Image compression based on matrix factorization of Satellite images. (i)
Original image, (ii).Reconstructed image based on Non-iterative matrix factorization, (iii)
Reconstructed image based on iterative matrix factorization
The comparative analysis of the LRMSE and the processing time values are
listed in Table.2.1. It is seen that the quality of the reconstructed images using positive
matrix factorization is visually lossless. For comparison purpose the non-iterative
method of matrix factorization is compared with iterative method of matrix factorization.
Table 2.1: Performance analysis based on intensity based factorization
Non-iterative Matrix
Iterative Matrix
Factorization
Factorization
category
Image
Processing
LRMSE
satellite image
images
images
microscopic
Electron-
Medical
time(sec)
Processing
LRMSE
time(sec)
mri1.png(512X512)
1.45
3.405
232.654
3.722
xray.tif(760X1152)
3.02211
1.8173
269.8475
8.5165
us1.tif (512X512)
5.852751
2.022
284.121
6.305
metal1.tif(532x576)
3.146
3.1478
273
6.7371
metal2.tif(748x1000)
7.680919
2.5599
290.765
5.761
metal3.tif(280x480)
1.44987
3.2735
229.989
5.245
urban area 1.tif
10.39571
6.0691
930.772
8.503
10.34532
6.5995
928.356
9.678
10.2801
2.5666
928.856
9.432
(1024x1024)
urban area 2.tif
(1024x1024)
urban area3.tif
(1024X1024)
2.5 Conclusion
Although the reconstructed images are visually lossless, it is worthwhile to
consider possible ways and means for further improving the quality of the reproduced
images with more reduction in LRMSE values. In this regard, a number of transform
methods have been considered and the findings are reported in the next chapter.