International Journal of Applied Engineering Research ISSN 0973-4562 Volume 11, Number 23 (2016) pp. 11582-11588
© Research India Publications. http://www.ripublication.com
Video Acquisition Device Identification using Sensor Pattern Noise
Sang-Hyeong Lee1, Dong-Hyun Kim2 and Hae-Yeoun Lee3*
Graduate Student, 2Graduate Student, 3PhD, Professor
1
1-3
Department of Computer Software Engineering, Kumoh National Institute of Technology,
1 Yangho, Gumi, Gyeongbuk, 39177, Republic of Korea.
(*Corresponding Author)
Abstract
Multimedia forensics are required to protect contents against
illegal usage. This paper presents a multimedia forensic
algorithm for video to identify the devices used for acquiring
unknown video files. As a unique fingerprint for each sensor,
the sensor pattern noise (SPN) is extracted and used for
designing a device identification algorithm. First, the ways to
calculate a SPN using DWT, Wiener filter, Gaussian,
Morphology are explained. Then, the ways to identify the
video acquisition device of the unknown video are presented.
Through intensive experiments using 30 devices, the
performance of video acquisition device identification
algorithms using SPNs from DWT, Wiener filter, Gaussian,
Morphology are analyzed and the DWT-based algorithm can
achieve the 98% identification accuracy. Different algorithms
also can achieve similar or less performance to the DWTbased algorithm.
Keywords: Multimedia Forensics, Sensor Pattern Noise,
Video Acquisition Device Identification.
INTRODUCTION
Internet technologies have rapidly developed and as results
they are many social services such as Facebook, KakaoTalk.
In addition, the number of users is growing and that internet
technologies occupy a part of our lives. Many multimedia
contents have been produced through social services using
multimedia contents acquisition devices, which is rapidly
increasing. Acquisition devices including Smart phone, digital
camera, DSLR, action CAM and some other devices produce
multimedia contents. Acquisition device has developed in a
form of high performance and high quality.
Multimedia devices and software can be easily accessed by
everyone with low cost, high quality and performance. Since
novice uses them for illegal purposes, many crimes are
happening which becomes a critical social issues. In most
crimes, videos from CCTV and car black box are referred to
solve cases and also adopted as an evidence. However,
multimedia is exposed to forgery and that can cause serious
social and illegal problems. Therefore, a technique to protect
the illegal usage of multimedia is required, and multimedia
forensics can be an effective solution to protect contents [1][2].
This paper focuses on respect to a technique for determining
the acquisition of a digital video as a multimedia forensics
technique. Most image acquisition devices have the sensor.
The sensor has a unique noise component because the
manufacturing process is not perfect. Therefore, this noise has
an advantage as the value of the unique fingerprint on the
device in forensics. Thus, finding a unique trace of the sensor
closely is related to the performance of multimedia forensic
techniques. The study on the various video devices has been
in progress in order to find a unique trace of accurate sensors.
In this paper, the video acquisition device identification
algorithm is presented with the sensor pattern noise extracted
by using DWT, Wiener Filter, Gaussian, Morphology filtering.
The ways to extract the sensor pattern noise is described,
where digital video characteristics that is composed of the
majority of frames are utilized. The description of the video
acquisition device identification process using the sensor
pattern noise is presented. First, the sensor pattern noise of a
reference device is extracted by the reference video and the
sensor pattern noise is extracted by the test video. Then, the
similarity calculation between two sensor pattern noises is
calculated to determine the acquisition device. To analyze the
performance of the proposed technique, 30 devices including
DSLR, compact camera and smart phone were used.
Quantitative analysis was performed to measure the
performance.
This paper is composed of 5 sections as follows. Section 2
explains the summary of research on the multimedia forensics
technology. Section 3 proposes a digital imaging acquisition
devices identification technology using sensor pattern noise
and explains the sensor pattern noise extraction algorithm.
Section 4 presents the analyzed results and Sec. 5 will
conclude.
RELATED WORKS
The performance of multimedia technology depends on the
way to extract accurately the unique features that are
presented in the digital content. Multimedia forensics
technology has been primarily studied in the institutes funded
by United States Governments or Air Force Research
Laboratory. These technologies can be largely divided into
two categories: imaging source identification technique and
forgery detection technique.
11582
International Journal of Applied Engineering Research ISSN 0973-4562 Volume 11, Number 23 (2016) pp. 11582-11588
© Research India Publications. http://www.ripublication.com
Fridrich et al. have conducted a study to identify the recording
device by using a specific pattern noise (Photo Response NonUniformity: PRNU) of the camera caused unintentionally by
the sensor in the imaging process [3, 4]. Memon et al are
conducting research to identify the shooting device. They use
the correlation between the pixels caused by the properties
and color filter array interpolation generated in the image
capture process [5].
Farid et al. have studied the technique to identify the forgery
of contents using the skewness-camera parameters, calculating
a space-time correlation of the image, or analyzing the
statistical properties of the video compression format [6, 7].
Lee et al. have performed the study to identify the imaging
device such as analyzing the sensor pattern noise of CCTV
imaging techniques to identify the device to shoot a picture,
techniques to identify whether the operation of the image,
forgery detection techniques with a change in the color filter
matrix. It is capable of determining the photographing
apparatus in the image size conversion techniques studies
[1,8,9].
PRNU using digital forensic techniques to identify the image
capture device is a tendency to develop steadily or constantly.
Digital forensic techniques relating to forgery detection
technique have been continuously studied to find a form of the
statistical properties of the image.
Stamm et al. proposed the detecting method for identifying
the device using the amplification of the image contrast[10].
To identify a unique fingerprint of the specific characteristics
of the histogram, identify the use form of the histogram
equalization and contrast improvement can be used to
determine whether or not there is a falsification of the image.
Mahdian et al. studied a method of detecting a mark by adding
random noise to the local image area and this has made
commonly changes that are used to cover the signs of
modulation[11].
PROPOSED
VIDEO
IDENTIFICATION
ACQUISITION
DEVICE
This section explains multimedia forensic techniques to
determine the video acquisition device using the sensor
pattern noise. Implementation process of the algorithm for
determining the video acquisition device is shown in Fig. 1.
Figure 1. Video Acquisition Device Identification Process
First, it extracts frames from acquisition video from the
reference device. The sensor pattern noise estimates from the
extracted frame using filtering algorithm. After removing the
periodic noise, reference sensor pattern noise calculates from
the reference video acquisition device. The sensor pattern
noise extracted from each frame performs averaging operation.
Sensor pattern noise of unknown video extracts in a similar
way. Second, by comparing the similarity of two sensor
pattern noise, it may determine the video acquisition device.
When determining to match, we measure normalized
correlation coefficient (NCC) value of each sensor pattern
noise and if NCC is higher than specific threshold, unknown
video is judged by reference video acquisition device.
Otherwise, it can be determined irrelevant.
Noise Extraction with Filtering Algorithm :
As the unique characteristics of imaging sensors, sensor
pattern noise based on noise filter is extracted. In the first step,
noises from each frame are extracted by applying filtering
algorithm and then all extracted noises are averaged as
follows.
2
N
N
K = ∑M
i=1(∑k=1 Wi,k Ii,k / ∑k=1 Ii,k )
(1)
Where W = I - WF (I), WF is Wiener filter, N is the number
of frames in each video. For the test frames of the unknown
video, noise filter can be extracted in similar to the noise of
reference videos. However, only 1 video is considered.
In this paper, we have applied 4 filtering algorithms for
extracting noise in images: discrete wavelet transform-based
algorithm, wiener filtering based algorithm, Gaussian filtering
based algorithm and morphology-based algorithm. Details of
these algorithms will be explained.
11583
International Journal of Applied Engineering Research ISSN 0973-4562 Volume 11, Number 23 (2016) pp. 11582-11588
© Research India Publications. http://www.ripublication.com
A. Discrete Wavelet Transform
When the sensor pattern noise is being estimated, it is
important to reliably estimate the noise included in the frame
of each video. In this section, it explains the algorithm for
estimating the sensor pattern noise using the discrete wavelet
transform (DWT).
The sensor pattern noise can be calculated using the difference
of the frame which removes the noise from the original frame.
The noise estimation method used a discrete wavelet
transform to apply the discrete wavelet transform with respect
to the frame. When applying a discrete wavelet transform, the
frame is divided into four sub-bands as shown in Fig. 2.
Among transformed coefficients, Wiener filtering applies on
the LH, HL, HH coefficients, except for LL where is the lowfrequency. As a result, it is possible to eliminate noise
contained in the high-frequency areas other than the low
frequencies. Thus, the noise is displayed in the high-frequency
component. When restoring the frame after removing the
noise contained in the LH, HL, HH, we may obtain an image
in where sensor pattern noise has been removed.
Figure 3. 2-Dimensional Gaussian Distribution
C. Wiener Filter
Wiener filter effectively remove the noise included in the
image. Wiener filter has the form that minimizes mean-square
error filter. The formula to apply in Wiener filter is as follow
Iw (x, y) = μ +
σ2 −v2
σ2
(I(x, y) − μ)
(5)
I: is original image. Iw : is image applied to wiener filter. μ: is
the regional average for each pixel. σ : is the local variance for
each pixel. v: is the variance of the noise. If you do not know
the variance of noise, the variance of noise is applied to the
average value calculated from the variance which is estimated
for each pixel.
D. Morphology
Figure 2: 4-Level Discrete Wavelet Transform
B. Gaussian Filter
Gaussian filter is used in the image processing by using a
value of the calculated Gaussian distribution as a mask.
Gaussian filter has the effect of removing noise caused by a
probability distribution and a normal distribution. 2dimensional Gaussian distribution is shown in Fig. 3.
Formula of Gaussian distribution is as follow.
G(x, y) =
1
2πσ
2 exp
−(x2 +y2 )
2σ2
(4)
Mask value is calculated using Gaussian distribution. Mask
size set 15 and σ set 7.0. Thus, sensor pattern noise is obtained
from Gaussian distribution.
Morphology is a method to access the image morphologically.
Morphology is an image processing technique that is used for
the purpose of modifying the shape of the specific object
existing in the image. This is used for binary images and
grayscale images. Morphology is used for mathematical
characteristics such as the containment, movement, symmetry,
a complementary set, a difference set.
Morphology operation performs erosion, swelling, open and
closed operations on the gray scale image. Erosion operation
is carried out on an operation in the form to increase the dark
area by reducing the light areas in the gray scale image. The
expanded operation is carried out on an operation in the form
to increase the light area by reducing the dark areas in the
gray scale image. When applying the morphology operation
with respect to gray-scale image, it may detect the pixels that
the surrounding pixels with a difference due to noise. It is
possible to take advantage of the sensor pattern noise
estimation. Fig. 4 presents an extracted noise.
11584
International Journal of Applied Engineering Research ISSN 0973-4562 Volume 11, Number 23 (2016) pp. 11582-11588
© Research India Publications. http://www.ripublication.com
Figure 4. Noise Extraction with Morphology Filtering
Periodic Noise Removal
The accuracy of this averaged noise is low because of block
effects from 8x8 block or macro-block during MPEG
compression. In the second step, since these effects have
periodic characteristics, Fourier transform is performed and
Wiener filtering is applied to remove these block effects and
noises which is shown in the formula below
K` = F −1 {F(K) − W(F(K))}
(2)
where F is Fourier transform and W is Wiener filtering. The
extracted reference noise is K’.
Similarity Comparison
Sensor pattern noise that is extracted from reference video
acquisition device is k`. Sensor pattern noise that is extracted
from unknown video is T` . Normalized cross correlation
(NCC) value that presents the similarity of two sensor pattern
noises is calculated as follow:
corr(K`, T`) =
̅ )×(T`−T`
̅)
(k`−k`
̅ ‖×‖T`−T`
̅‖
‖k`−k`
(3)
When the calculated NCC value is over a pre-defined
threshold, it means that the unknown video is acquired using
the reference video acquisition device.
In order to determine this threshold, we have calculated NCC
values from un-correlated videos. About 1,305,000 frames of
300 videos are considered. Then, the NCC values are fitted by
Gaussian distribution model as shown in Fig. 5. Then, the
value having 1/106 error probability is selected as the
threshold. As a result, the threshold value for DWT-based
filtering, wiener filtering, Gaussian filtering, and morphologybased filtering is 0.0071, 0.0068, 0.0076, and 0.0083,
respectively.
Figure 5. NCC distribution and its Gaussian fitting model
11585
International Journal of Applied Engineering Research ISSN 0973-4562 Volume 11, Number 23 (2016) pp. 11582-11588
© Research India Publications. http://www.ripublication.com
EXPERIMENTAL RESULTS
30 video acquisition devices from 14 brands are considered to
analyze the performance of the algorithm. Without any special
setting for each device, 5 videos about 10 sec were taken to
estimate the reference sensor pattern noise of the reference
devices. 5 videos about 10 sec were taken for testing
identification accuracy. To analyze the identification accuracy,
the sensor pattern noise using each algorithm from unknown
videos are extracted and compared with reference sensor
pattern noise using each algorithm from 5 reference videos of
each reference imaging device. The device having high NCC
over the threshold is considered as the device used to acquire
the unknown video. The identification rate for each device is
summarized in Table.1 and Fig. 6.
Table 1. Identification Results Using 4-Algorithm
Brand
Model
Canon
Canon 650D
Canon EOS 500D
Canon EOS M
Canon EOS M3
Canon IXUS 160
Nikon Coolpix S100
5
5
5
5
5
5
0
0
0
0
0
0
5
5
5
5
5
5
0
0
0
0
0
0
5
5
5
5
5
5
0
0
0
0
0
0
5
5
5
5
5
5
0
0
0
0
0
0
Nikon Coolpix S33
Panasonic Panasonic DMC SZ1
Lumix DWC LX100
Olympus Olympus PEN Mini
Samsung Samsung WB35F
Samsung NX Mini
5
5
5
5
5
5
0
0
0
0
0
0
5
5
5
5
5
5
0
0
0
0
0
0
5
5
5
5
5
5
0
0
0
0
0
0
5
5
5
5
5
5
0
0
0
0
0
0
Sony
Samsung
GoPro
LG
Sony HDR XR520
Galaxy Note3
GoPro Hero4
G2
G3 Cat6
G4
Vu3
5
5
5
5
5
3
5
0
0
0
0
0
2
0
5
5
5
4
5
3
5
0
0
0
1
0
2
0
5
5
5
4
5
4
5
0
0
0
1
0
1
0
5
5
5
5
5
3
5
0
0
0
0
0
2
0
Samsung Galaxy Grand Max
Galaxy Note2
Galaxy Note2
Galaxy Note4
Galaxy S4
Galaxy S5
5
5
5
5
5
5
0
0
0
0
0
0
5
5
5
5
5
5
0
0
0
0
0
0
5
5
5
5
5
5
0
0
0
0
0
0
5
5
5
5
5
5
0
0
0
0
0
0
Galaxy Zoom2
Huawei P8 GRA UL00
Shaomi MI Note LET
Apple
iPhone 6plus
Pantech Vega Secret Note
Average Accuracy
5
5
5
4
5
0
0
0
1
0
5
5
5
3
5
0
0
0
2
0
5
5
5
3
5
0
0
0
2
0
5
5
5
4
4
0
0
0
1
1
Nikon
DWT
Wiener Filter
Gaussian
Morphology
Success Failure Success Failure Success Failure Success Failure
98%
As a result of the above, these results confirmed that the
presented algorithm could perform well and can be used to
protect multimedia for illegal usage. It determines the video
acquisition device regardless the type of device. And among
the 4-algorithms, DWT has the best performance. It was tested
96.6%
97.3%
97.3%
using the same image 4-algorithms. As feature of the
algorithms have different results. All algorithms have a high
performance. Therefore, using any algorithm you can achieve
good convenient results.
11586
International Journal of Applied Engineering Research ISSN 0973-4562 Volume 11, Number 23 (2016) pp. 11582-11588
© Research India Publications. http://www.ripublication.com
But, such as iPhone 6Plus, devices equipped with post
processing software do not perform accurate identification
because the software is done through the process of
converting a crisp and clean video, the sensor pattern noise is
modified.
CONCLUSION
Development of the Internet and computer technology has
made the setting for acquiring and distributing high-quality
multimedia content. However, due to the user who can use it
for illegal purposes, multimedia content has been
indiscriminately spilled. Accordingly, a problem of copyright
and privacy is generated.
In this paper, we proposed a digital video acquisition device
identification technique with respect to use a sensor pattern
noise. After explaining the technique for detecting the sensor
pattern noise, we extract the sensor pattern noise on the
reference image and the sensor noise on the test pattern image.
Then, that digital video acquisition device identification
technique proposed a method to identify the digital imaging
device through a calculation of similarity between the two
noises. For the efficient analysis of the proposed techniques
and by using a variety of media content imaging device was
performed as an experiment, it was confirmed to show an
excellent performance.
Our proposed technique, first, the illegal image authors
estimated through video acquisition device identification
technique. Second, the integrity of the court evidence, such as
CCTV video proved. Third, copyright of your images proved
and other various applications are possible.
In particular, unlike the watermarking technique of changing
the source by deliberately inserted. Digital forensic techniques
for analyzing the characteristics of the image itself, does not
bring a change in the source. In order to apply this technology,
it can be performed without changing the existing system. It
can be applied in various fields on the basis of these
advantages. Currently it is possible to detect a forgery picture
to deceive military technology. In addition, it is possible to
detect the forgery of the black-box image of the scene. In this
way, it is possible to detect a wide range of image forgery. It
is expected that future studies will be conducted actively in
this direction
Figure 6. Video acquisition device identification result
11587
International Journal of Applied Engineering Research ISSN 0973-4562 Volume 11, Number 23 (2016) pp. 11582-11588
© Research India Publications. http://www.ripublication.com
ACKNOWLEDGEMENT
This work was supported by Kumoh National Institute of
Technology.
REFERENCES
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
Hyun, D.-K., Ryu, S.-J., Lee, H.-Y., Lee, H.-K.,
2013, “Detection of Upscale-Crop and Partial
Manipulation in Surveillance Video based on Sensor
Pattern Noise”, Sensors 13(9), 12605-2631
Li, C.-T., 2010, “Source Camera Identification Using
Enhanced Sensor Pattern Noise”, IEEE Transactions
on Information Forensics Security, 5(2), pp. 280-287
Lukas, J., Fridrich, J., and Goljan, M., 2006, "Digital
camera identification from sensor pattern noise",
IEEE Transactions on Information Forensics Security,
1(2), pp. 205–214.
Chen, M., Fridrich, J., and Goljan, M., 2007, "Source
digital camcorder identification using ccd photo
response non-uniformity", Proceedings of SPIE
Electronic Imaging, Security, Steganography, and
Watermarking of Multimedia Contents IX, San Jose,
CA, pp. 1G–1H.
Bayram, S., Sencar, H. T., Memon, N., and Avcibas,
I., 2005, "Source Camera Identification Based on
CFA Interpolation," Proceedings of IEEE
International Conference on Image Processing (ICIP),
pp. 69–72.
Popescu, A. C. and Farid, H., 2005, “Exposing
Digital Forgeries by Detecting Traces of
Resampling”, IEEE Transactions on Signal
Processing, 53(2), pp. 758–767.
Wang, W. and Farid, H., 2006, "Exposing Digital
Forgeries in Video by Detecting Double MPEG
Compression", Proceedings of the 8th workshop on
ACM Multimedia and security, pp. 37-47.
Choi, C. –H., Lee, H.-Y., and Lee, H.-K., 2013,
"Estimation of Color Modification in Digital Images
by CFA Pattern Change", Forensic Science
International, An International Journal, Elsevier,
226(1-3), pp. 94-105.
Lee, J.-W., Lee, M.-J., Lee, H.-Y., and Lee, H.-K.,
2012, "Screenshot Identification by Analysis of
Directional Inequality of Interlaced Video",
EURASIP Journal on Image and Video Processing,
Springer, 2012(7), pp. 1-15.
Stamm, M. C. and Liu, K. J. R., 2010, “Forensic
Detection of Image Manipulation Using Statistical
Intrinsic Fingerprints,” IEEE Transactions on
Information Forensics and Security, 5(3), pp. 492506.
[11]
Mahdian, B. and Saic, S., 2009, “Using Noise
Inconsistencies for Blind Image Forensics,” Image
and Vision Computing, 27(10), pp. 1497-1503.
11588