Spectral Multiplexing of QR Codes
Tiago Nuno Pereira Mota
Thesis to obtain the Master of Science Degree in
Electronics and Computer Engineering
Supervisor: Prof. Paulo Sérgio de Brito André
Examination Committee
Chairperson: Prof. José Eduardo Charters Ribeiro da Cunha Sanguino
Supervisor: Prof. Paulo Sérgio de Brito André
Members of the Committee: Prof. Gonçalo Nuno Marmelo Foito Figueira
November 2015
Resumo
Com o constante desenvolvimento da utilização de smartphones, um dos principais meios de transmitir
informação usando estes dispositivos, o código QR, sofreu um aumento significativo. Estes códigos de
barras bidimensionais permitem transmitir uma grande quantidade de informação de forma acessível
através de um smartphone.
Nesta dissertação será estudado o aumento de capacidade do código QR, recorrendo a técnicas de
multiplexagem espectral.
Após uma análise inicial do potencial ruido que possa perturbar a leitura dos códigos QR, realizada em
40 códigos Qr diferentes, procedeu-se a utilizar uma técnica multinível para a multiplexagem, utilizando
os bits do código QR para os níveis em escala preta. Esta técnica é ainda utilizada para os espaços de
cor RGB, realizando-se a multiplexagem espectral de códigos QR.
Visto se trabalhar com os espaços de cor RGB em ambas as técnicas mencionadas, a possibilidade de
uma junção de ambas as técnicas enunciadas foi considerada e testada em 40 códigos QR, verificandose ser a melhor das técnicas enunciadas para transmissão de elevada quantidade de dados, permitindo
transmitir dados de imagens de dimensões 50x50 e 1,9 KB com facilidade e um ficheiro de som de 23,2
KB com a duração de um minuto e meio com uma relação sinal-ruído muito baixa de 36,03 dB.
Palavras-chave:
códigos QR, multiplexagem espectral, RGB, multinível, binário
i
ii
Abstract
With the constant development of utilities in smartphones, one of the main means of transferring
information using these devices is QR codes. These bidimensional barcodes allow for a great quantity
of information to be transferred in an accessible way through a smartphone.
In this thesis, a study in order to increase of the data capacity of QR codes will be conducted, using
spectral multiplexing techniques.
After an initial analysis of the potential noise that may disturb the reading of QR codes, the possibility of
using a multilevel technique is considered for multiplexing, by using the bits from the QR codes as the
levels in grayscale, and then for the other RGB color spaces in order to perform spectral multiplexing of
QR codes.
Since both techniques used the RGB color spaces, the possibility for using both techniques at the same
time was considered and testes, demonstrating to be the best of the three, allowing for the data of
images of size 50x50 and 1,9 KB to be transferred with ease and a 23,2 KB audio file to be transferred
with almost no noise in it by having a SNR of 36,03 dB.
Keywords:
QR codes, spectral multiplexing, RGB, multilevel, binary
iii
iv
Index
Resumo.................................................................................................................................................... i
Abstract .................................................................................................................................................. iii
Index ........................................................................................................................................................v
Figures list ............................................................................................................................................ vii
Tables list ............................................................................................................................................... ix
Acronyms list ......................................................................................................................................... xi
1. Introduction ........................................................................................................................................ 1
1.1 Motivation ......................................................................................................................................... 1
1.2 State of the art ................................................................................................................................. 3
2. Theory ................................................................................................................................................. 7
2.1 The QR code .................................................................................................................................... 7
2.2 Noise in images ............................................................................................................................. 16
2.3 Color spaces .................................................................................................................................. 18
3. Implementation ................................................................................................................................ 21
3.1 Noise ............................................................................................................................................... 21
3.2 Multilevel quick response codes ................................................................................................. 24
3.3 Multiplexing with color coding ..................................................................................................... 29
3.4 Multilevel color spectral multiplexing ......................................................................................... 33
4. Large capacity QR Code ................................................................................................................. 35
4.1 Image transfer ................................................................................................................................ 35
4.2 Audio transfer ................................................................................................................................ 39
5. Conclusions ..................................................................................................................................... 43
References ........................................................................................................................................... 45
v
vi
Figures list
Fig. 1 Example of QR code being used in advertisement ....................................................................... 1
Fig. 2 Examples of an UPC-A barcode and a barcode in patient wristband ........................................... 2
Fig. 3 Example of a 3D code based of on a QR code ............................................................................. 3
Fig. 4 UPC barcode and a EAN8 barcode .............................................................................................. 4
Fig. 5 A typical Code 39 barcode ............................................................................................................ 5
Fig. 6 Data matrix barcode in its square form ......................................................................................... 5
Fig. 7 Example of an Aztec barcode ....................................................................................................... 6
Fig. 8 QR code encoding process ........................................................................................................... 8
Fig. 9 Pattern positions in a QR code .................................................................................................... 12
Fig. 10 Data bit placement in a QR code .............................................................................................. 13
Fig. 11 QR code decoding process ....................................................................................................... 16
Fig. 12 Gaussian noise in a QR code produced in Matlab for variance = 0,1 ....................................... 17
Fig. 13 Salt-and-pepper noise in a QR code produced in Matlab for noise density = 0,2 ..................... 17
Fig. 14 Shot noise in a QR code produced in Matlab ............................................................................ 18
Fig. 15 RGB color spaces...................................................................................................................... 18
Fig. 16 Three dimensional representation of RGB color space ............................................................ 19
Fig. 17 Digital image color flow [11] ...................................................................................................... 19
Fig. 18 Speckle noise in QR codes for 250 characters for L error correction level and M error
correction with variance 0.04, 0.2 and 0.3 ......................................................................................... 22
Fig. 19 Shot noise (Poisson) in QR codes for 250 characters for L and M error correction levels ....... 23
Fig. 20 Salt and Pepper noise for 250 characters for an L error correction level with noise density of
0.1, 0.2 and 0.7 respectively ............................................................................................................. 23
Fig. 21 Zero Mean White Gaussian noise with 0.01 variance for 250 and an L and M error correction
levels for 250 characters ................................................................................................................... 23
Fig. 22 Multilevel intensities for a grayscale QR code [10] ................................................................... 24
Fig. 23 Multilevel intensities for color channels in the conventional QR code [10] ............................... 24
vii
Fig. 24 Example of a colored QR code using the method described [10] ............................................. 25
Fig. 25 Examples for colored QR codes using a single color for all the codes ..................................... 26
Fig. 26 QR codes in blue color space ................................................................................................... 28
Fig. 27 QR codes in blue color space after being multiplied by their respective values ....................... 28
Fig. 28 "Adding” of previous QR codes ................................................................................................. 28
Fig. 29 Multiplexing and color coding table for 3 QR codes [3] ............................................................. 30
Fig. 30 Examples of multiplexing 2 QR codes for different color spaces .............................................. 31
Fig. 31 QR color code using the first code for red and blue and second code for green ...................... 31
Fig. 32 Multiplexing of 3 QR codes into one where each code has a different color space ................. 32
Fig. 33 Example of multiplexing 6, 9 and 12 different codes into one ................................................... 33
Fig. 34 50x50 image and QR code holding its information ................................................................... 36
Fig. 35 Version 18 QR code holding image information ........................................................................ 37
Fig. 36 Version 40 QR code holding data file information ..................................................................... 40
Fig. 37 Audio file and reconstructed audio file ...................................................................................... 41
viii
Tables list
Table 1 Data capacity (characters) for different versions of QR codes .................................................. 7
Table. 2 Binary indicator for existing modes ........................................................................................... 9
Table 3 Number of bits required by each mode and version .................................................................. 9
Table. 4 Masking patterns for QR codes ............................................................................................... 14
Table. 5 Bits associated with each Error Correction Level .................................................................... 15
Table. 6 Letter frequency for the english language ............................................................................... 21
Table 7 Average values for when the QR code becomes unreadable .................................................. 22
Table 8 QR code values for amount of codes being multiplexed .......................................................... 27
Table 9 Possible color combinations for a color QR code .................................................................... 29
Table 10 RGB values for 3 QR codes being multiplexed ...................................................................... 38
ix
x
Acronyms list
AIAG - Automotive Industry Action Group
CCD – Charge Coupled Device
EIA - Electronic Industries Alliance
EC – Error Correction
EAN – European Article Numbering
ICC – International Color Consortium
CIE - International Commission on Illumination
OCR – Optical Character Recognition
1D – one dimensional
QR – Quick Response
RGB – Red Green and Blue
TIFF/EP - Tagged Image File Format / Electronic Photography
2D – two dimensional
UPC – Universal Product Code
xi
xii
1. Introduction
1.1 Motivation
With the growth in today’s society, where almost anyone has a smartphone with an internet connection,
companies strive to outdo each other and find a way to gain consumers. One of the ways they came up
with, was by taking advantage of the smartphones apps and camera to act as a barcode reader. With
this technology, companies started using QR codes in posters to help divulge their products or as
promotional giveaways much like the food stamps you would get by visiting fast food places like
Burguerking or Pans, and so the popularity of these codes skyrocketed.
Fig. 1 Example of QR code being used in advertisement
But even with the high capacity to hold data that these codes provide, its true potential has not yet been
reached and researchers strive to develop better QR codes that can hold even greater amounts of data
by using several methods that will be explored in this thesis.
But it’s not just advertisement companies that have an interest in barcodes. Due to the versatility of
barcode technology, one dimensional codes have a wide use in society, such as in the case of
manufacturing companies or the healthcare industry. In the case of the manufacturing industry, barcodes
will provide real-time information on products thus improving product stock management by being able
to identify and track products without the need to access a database, or by being to perform quality
control and testing on said products [1, 2].
In healthcare, error and resultant malpractice minimization control can be done by monitoring which
medicines and equipment was used on the patient, and by whom, where and when it was used, by
having a barcode with said information on the patients wristband as seen in fig.2 . Barcodes also allow
for inventory control and replenishment, traceability systems and faster and more accurate billing [2].
1
Fig. 2 Examples of an UPC-A barcode and a barcode in patient wristband
With the advance in today’s technology, their use has expanded greatly into the publicity area by
upgrading from 1D codes to 2D codes such as QR codes.
These barcodes were developed by the Denso Wave Incorporated in 1994 as a means to transmit
information through a print-scan channel for the automotive industry and were approved as an ISO
international standard (ISO/IEC18004) in June 2000 [1, 2, 3].
In recent years their use has expanded to product tracking, item identification, time tracking, document
management as well as general marketing, due to its simplicity and compatibility with smartphones [4].
Driven by the development in camera phones, these smartphones can work as scanners, barcode
readers and portable data storages with network connectivity, which when used with a 2D barcode allow
it to work as a bridge between the physical world and the digital world, such as allowing consumers to
purchase online e-tickets and simply using them at the counter instead of waiting in lines by having the
QR code on their phone that contains their ticket information read by a scanner [2].
It is clear that QR codes have a great uses be it in healthcare, the manufacturing industry or even the
advertisement industry, but are limited by their capacity. Therefore the objective of this work is to study
how to implement spectral multiplexing techniques such as multilevel or color coding, in order to
increase the capacity that the QR codes currently possess and demonstrate that with this increase in
capacity that the QR codes could potentially transmit other kinds of data than just text, such as images
or even audio files, which could open up new possibilities for the advertisement or even the music
industry.
2
Fig.2 retrieved from https://upload.wikimedia.org/wikipedia/commons/5/5d/UPC-A-036000291452.png and
https://upload.wikimedia.org/wikipedia/en/a/ad/LB2-ADULT-L3_Assembled.jpg
1.2 State of the art
Barcodes are considered a machine-readable representation of information formed by the combinations
of high and low reflectance regions at the surface of an object, which can be converted into a binary
representation. In the initial version, information was encoded into an array of adjacent bars and spaces
with varying width, thus being called “barcodes”. The first barcodes designed were the one-dimensional
ones, where the code was read by a scanner that swept the code with a beam of light in a straight line
[2].
By switching the bars and spaces with dots and spaces arranged in an array or matrix, we can increase
the density of data within space, resulting in a two-dimensional barcode. Obviously this means that
unlike the 1D code, the 2D ones will require a scanner that can read vertical and horizontal directions.
We can find also a three-dimensional coding that is often used in irregular surfaces. The code uses the
‘bars’ from the 1D codes or the ‘dots’ of the 2D codes and embosses them on the surface of the material.
The 3D barcodes use special elements to generate high and low reflectance instead of a visual contrast
between different colors as seen in fig.3 [2]. This method would allow a further increase in the capacity
of the previous codes since for different layers of thickness we could possibly transmit additional
information. It is to note that mobile scanners such as the ones in telephones do not currently have the
capacity to read these codes.
Fig. 3 Example of a 3D code based of on a QR code
Being one of the several types of automatic identification and data capture systems used nowadays,
alongside RFID systems, optical character recognition (OCR) Systems and magnetic stripe technology,
barcode technology was originally developed around the 1800s being used at first as a reading aid for
the blind [2].
Around the 1950s thanks to the technological advancement in computing, data recognition patterns
were developed, where a patent by the name of ‘Classifying apparatus and method’ was filed by Norman
3
Fig.3 retrieved from http://www.mobiliodevelopment.com/wp-content/uploads/2012/03/3D-QR.jpg
J. Woodland and Bernard Silver composed by a series of concentric circles instead of in straight lines
used for 1D codes [2].
This patent was the first step towards what we have as the current linear barcodes, and at that time,
one of its creators, Woodland, thought of a barcode using vertical lines instead of circles, but decided to
use the concentric circles since it allow omnidirectional scanning [2].
Nevertheless, the early stages of barcode symbology was difficult due to inadequate pattern reading
and scanning technology, since only transducers were capable of scanning patterns at the time, unlike
the ones we have currently that use pen and flashlight-type light sources and mirrors, but thanks to the
inventions of the moving-beam laser scanner in the 1960s, and of the charge coupled device (CCD)
scanner and pixel readers, further development of barcode systems became possible.
The first adopted linear barcode was called Universal Product Code (UPC), where it was considered
that all the interest groups would use the same symbol structure, allowing for each code to be uniquely
identified [2].
Due to foreign interest by the European Article Numbering Association in UPC, it was adopted as the
European Article Numbering (EAN) code format as seen in fig.4. This code could be omnidirectionally
read, was compatible with UPC, did not require special printing and wrapping and could be decoded at
a speed within 100 inches per second [2].
Fig. 4 UPC barcode and a EAN8 barcode
The difference between these 2 formats was that while the UPC was considered for use only in the
United States, the EAN format was considered for a global use and so it includes two or three
identification numbers as a prefix that identifies the GS1 Member Organization. The EAN format was
adopted by more than a hundred countries and so is considered the global standard one dimensional
barcode.
The military of the USA adopted the barcode ‘Code 39’, seen in fig.5, for logistics applications of
automated marking and reading symbols (LOMARS),due to its ability to encode alphanumeric data it
became one of the most used industrial barcodes, being mandated by the Automotive Industry Action
Group (AIAG) and the Electronic Industries Alliance (EIA) for labelling [2].
4
Fig.4 retried from https://www.oclc.org/content/dam/oclc/bibformats/images/upc_code.jpg
https://upload.wikimedia.org/wikipedia/commons/thumb/f/f1/EAN8.svg/738px-EAN8.svg.png
Fig. 5 A typical Code 39 barcode
Since the 1D barcodes were introduced, there has been a growing demand to encode more and more
data than what a 1D barcode could accommodate. The Toyota Motor Corporation, along with the Denso
Wave Inc. tried to devise a barcode symbol that was similar to the 2D barcodes, and proposed a code
called the Codabar [1, 2, 3].
The Codabar was used in a parts-movement system called Kanban, which meant a plate or a card with
necessary information for the production, in order to increase the efficiency of the management of parts
flowing along the production line.
In 1978 a project utilizing Kanban and 2D barcode like symbologies began, which lead to the
development of current societies 2D barcodes that could encode from 10 to 100 times more data than
their 1D counterparts, while also requiring less space when an identical set of data is encoded thanks
to their matrix format.
One of such 2D barcodes is the Data Matrix barcode as seen on the right of fig.6. Developed by RVSI
Acuity Cimatrix, this barcode ranges from an 8x8 size to a 132x132 size, capable of holding up to 2335
characters in alphanumeric or 3116 numbers, and 71 characters and 96 numbers in its rectangle form.
Fig, 6 Data matrix barcode in its square form
Another often used 2D barcode is the Aztec code. Invented by Andrew Longacre, this 2D barcode has
the potential to use less space than other matrix barcodes due to not requiring a blank ‘quiet zone’
around it. It ranges from 15x15 modules up to 151x151 by adding several layers of 2 pixels around the
core. This code can encode up to 3067 characters and 3832 digits.
5
Fig.5 retrieved from http://help.accusoft.com/BarcodeXpress/v9.0/activex/images/code39.fpg
Fig.6 retrieved from http//www.roemerind.com/wp-content/uploads/2010/07/Data-Matrix_Roemerind.jpg
Fig 7 Example of an Aztec barcode
As said before, 2D barcodes have many uses in healthcare, such as allowing elderly people to better
medicate themselves by giving them verbal instructions with the help of a voice app [5] and electronics
industries due to the need of handling small products [6], but that does not mean its usage is only in
those areas, as demonstrated by the automotive industry or even the security industry [7]
With the increase of capacity provided by 2D barcodes, they can now provide error detection and
correction capability, multilingual encoding (such as byte, alphanumeric, kanji) and different symbol
versions depending on the data required, allowing it to work as portable database, unlike the 1D
barcodes that work as an index to a backend database [2].
6
Fig.7 retrieved from http://www.mobiliodevelopment.com/wp-content/uploads/2013/06/aztec-code-2d-matrixbarcode.gif
2. Theory
2.1 The QR code
The QR code is composed by geometric patterns of black and white dots (modules), in a square white
background. The code has 3 finder patterns in the upper left, bottom left and top right corners that are
made from three overlapped homocentric squares of 7x7 black colored modules, 5x5 white colored
modules and 3x3 black colored modules [1, 3]. These finder patterns are used to define the position of
the seeking graph and obtain the image information from the reader.
The QR code also enables four type of encoding modes (numeric, alphanumeric, byte/binary and kanji)
which vary in capacity depending on the level of error correction and the version of the code.
As mentioned before, there are four levels of error correction by using Reed-Solomon error correction,
L (low), M (medium), Q (quartile), H (high) where 7%, 15%, 25% and 30% of symbols can be corrected
in case the original code has been damaged [1, 3]. This process creates error correction codewords
based on the data, which can be used by the reader to determine if the data was read correctly and if
so, use those same codewords to correct those errors.
Table 1 Data capacity (characters) for numeric, alphanumeric and kanji modes for different versions of QR codes
Version
1
5
6
7
40
Error
Correction
Level
Numeric
L
M
H
Q
L
M
H
Q
L
M
H
Q
L
M
H
Q
L
M
H
Q
41
34
27
17
255
202
144
106
322
255
144
106
370
293
207
154
7089
5596
3993
3057
Data Capacity
Alphanumeric
25
20
16
10
154
122
87
64
154
12
87
64
224
178
125
93
4296
3391
2420
1852
Kanji
10
9
7
4
85
52
37
27
65
52
37
27
95
75
53
39
1817
1435
1024
784
7
There are forty different versions for QR codes, going from 21x21 to 177x177 modules. The readers in
actual cellphones usually can only read up to the tenth version. These versions differ by stepping four
modules horizontally and vertically.
The encoding/decoding process is regulated by the norm ISO/IEC18004:2006 and is described in a
detailed way in the on-line website http://www.thonky.com/qr-code-tutorial/introduction.
The process of encoding of the QR code can be divided into six steps:
1. Data analysis, where the encoding mode most suited for the characters will be chosen;
2. Coding, where the Error Correction Level (ECL) will be chosen before the actual coding of the
message to its bit equivalent is performed so that the adequate QR code version will be used;
3. Error Correction, where the structure message will be divided into datablocks and where error
correction bits will be generated;
4. Arrangement, where the several patterns, modules, separators and the encoded message will
be placed in the QR code matrix;
5. Masking, where the QR code matrix will be changed to 8 different versions and submitted to a
scoring test to see which version is the best one;
6. Format information, where the error correction level and masking chosen will be placed in the
QR code matrix along with the QR code version.
Fig. 8 QR code encoding process
As was said, in the data analysis step the first thing to do is chose from one of four modes to encode
the message. The numeric mode as the name implies is used for decimal digits, that is “0,1,2,3,…,” ,
while alphanumeric mode also includes uppercase letters, symbols and a space. Byte mode is used if
the characters are not in the alphanumeric mode and Kanji mode is used for any characters belonging
to the Shift JIS character set. It is to note that while Shift JIS characters can be represented in UTF-8, it
can be possible to use byte mode instead of kanji mode to encode, but this requires three or even four
bytes while Shift JIS can be encoded in one or two bytes, thus kanji mode offers the highest capacity
for it. It is possible to use multiple modes in a single QR code by specifying the mode before the section
of bytes that uses said mode. QR codes also have difficulty accepting characters with punctuation such
as ’ã’ since these are not included in either alphanumeric or byte tables.
8
Before proceeding to the coding, one of the error correction levels must be selected, since a higher error
correction level will mean the QR code will have to be larger. After this is done, the version for the QR
code will be chosen based on the number of characters to be encoded and the error correction level
chosen.
We now need to add the mode indicator that identifies the mode select since the encoded data must
start with it in order to specify the mode being used for the bits. These mode indicators are presented in
the table 2.
Table 2 Binary indicator for existing modes
Mode Name
Mode Indicator
Numeric Mode
0001
Alphanumeric
Mode
0010
Byte Mode
0100
Kanji Mode
1000
A character count indicator will need to be added as well that will be placed after the mode indicator.
This indicator will essentially represent the number of characters being encoded and will have a specific
size depending on the QR version as shown below.
Table 3 Number of bits required by each mode and version
Mode
Version
1-9
10 - 26
27 - 40
Alphanumeric
10
12
14
Numeric
9
11
13
Byte
8
16
16
Kanji
8
10
12
Moving on to the encoding step, the message will need to be turned into its binary equivalent depending
on the encoding mode chosen. If the numeric mode was chosen, the digits will be split up into groups
of three and then convert them into binary. If it is alphanumeric, the characters will be split into groups
9
of two, converted into their number representation from the alphanumeric table, the first character will
be multiplied by 45 and then the second character will be added and the resulting sum will be converted
into binary. If byte mode is used, the characters will be converted into hexadecimal bytes and then into
an 8-bit string, padding it them on the left with 0’s to make it 8 bit long if necessary. If it is kanji the same
process as byte mode occurs but with the length of 13 bits instead of 8.
Each version and error correction level has a total number of bits required. In case the length required
wasn’t achieved after coding the message, the following steps will be performed until the length is
achieved:

add a terminator of up to four 0 bits at the right side of the string depending on how many bits
are left to reach the max length;

add 0 bits at the end of the string until it is a multiple of 8;

add 11101100 00010001 at the end of the string, repeating this final step until it reaches the
maximum length.
The error correction step begins by breaking the data codewords into groups and blocks depending on
the version and level chosen if necessary. This process begins by grouping the string into 8 bit parts
and converting then back into decimal. These numbers will them be used as the coefficients of the
message polynomial. After this is done, the generator polynomial will be chosen based on the version
and error correction level of the code, so that we can divide the message polynomial by it. This division
is based on the polynomial long division, considering the Galois field arithmetic.
The way polynomial long division works is:
1. take the divisor and multiply it by something that can make its first term equal to that of the
polynomial of the dividend;
2. subtract the new polynomial by the dividend;
3. if it is not possible for the remainder to be multiplied by an integer, the operation is complete,
else the process will repeat itself by using the remainder as the new dividend until this becomes
true.
Below is a very simple example of how a polynomial long division looks like by choosing 𝑥 2 + 𝑥 − 1 as
the dividend and 𝑥 + 1 as the divisor:
1. multiply 𝑥 + 1 by 𝑥, resulting in 𝑥 2 + 𝑥;
2. subtract 𝑥 2 + 𝑥 from 𝑥 2 + 𝑥 − 1, which leads to a remainder of -1;
3. since the remainder can no longer be multiplied by an integer the operation is complete.
It is necessary to note that the polynomial long term division is simpler in error correction since there is
no need to deal with the exponents of the polynomial terms.
10
Regarding the Galois Field, in QR code the bit-wise modulo 2 arithmetic and byte-wise modulo
100011101 arithmetic are used, which means we use Galois Field 28, also known as Galois Field 256.
The numbers contained in the Galois Field 256 range from 0 to 255, which is also the range of numbers
that an eight-bit byte covers, allowing for the mathematical operations performed in the Galois Field 256
to be represented as eight-bit bytes.
Negative numbers have the same values as positive numbers when in the Galois Field, so the absolute
value of the numbers are used when dealing with Galois Field arithmetic. Operations in the Galois Field
are cyclical, which means that if a result is higher than 255, it will require a modulo operation to get a
number that is in the Galois Field. Another trait that the absolute values of numbers have, is that
subtracting or adding is the same thing thanks to the modulo 2 operation performed afterwards which is
the equivalent of performing a XOR operation.
Returning to the byte-wise modulo 100011101, the QR code specifies that whenever a number is larger
than 256 it should be XORed with 285 that is, the decimal equivalent of 100011101, in order to get a
lower number than 256. This allows for all the numbers in the Galois Field 256 to be represented with
2n.
As said before in the polynomial long term division, 2 polynomials are required, which in this case will
be the message polynomial and the generator polynomial. The message polynomial uses the data
codewords obtained in the data encoding as the polynomial’s coefficients, while the generator
polynomial is created by multiplying(𝑥 − 20 ) … (𝑥 − 2𝑛−1 ), where n is the number of error correction
codewords that need to be generated.
After this is done begin the division steps below will be performed:

find the appropriate term to multiply the generator polynomial. The result should have the same
first term of the message polynomial or the remainder;

Apply XOR to the result of message polynomial or the remainder;

Repeat these steps x times depending on the x number of data codewords.
After the error correction codewords are found, it may be necessary to break up larger QR codes into
blocks that vary depending on version and error correction level. At this point the blocks will be
interleaved by taking the first data codeword from the first block, then the first from the second block
until it reaches the last block, where it returns to the first block and takes the second codeword and
repeats the process, until the last data codeword from the last block at which point the process is
repeated but for the error correction codewords.
The final message can now be convert back into binary in order to put the bits in the matrix for the
arrangement step. It is to note that some versions need additional bits called remainder bits, at the end
of the string after converting the final message back into binary.
11
Alignment Patterns
Finder Patterns
Timing Patterns
Separators
Dark Module
Fig. 9 Pattern positions in a QR code
Like it was said before the finder patterns are positioned at the top left, top right and bottom left corner
of the QR code and consist of an outer black square of 7 by 7 modules, an inner with square of 5 by 5
modules, and at last a black square in the center of 3 by 3 modules. This pattern was designed to be
unlikely to appear in any other sections of the QR code thus allowing the scanners to detect them and
orient the code for decoding.
Since the size of the code can be calculated by the formula (𝑉 − 1) ∗ 4 + 21, where V is the version of
the code, the top left finder pattern’s top left corner is always placed at (0,0), the top right finder pattern’s
top left corner is always placed at ((𝑉 − 1) ∗ 4 + 21 − 7, 0) and the bottom left finder pattern’s top left
corner is placed at (0, (𝑉 − 1) ∗ 4 + 21 − 7).
After adding the finder patterns, the separators can be added as well, which are lines of white modules
of only one module wide that are placed beside the finder patterns in order to separate them from the
rest of the code.
The alignment patterns are used on versions above version 1, and consist of a 5 by 5 black module
square, an inner 3 by 3 white module square and a single black module in the center. The locations of
these alignment patters are defined in the alignment pattern locations table where the numbers to be
used indicate the row and column coordinates for the center black module. It is to note that these
patterns cannot be put into the matrix before the finder patterns and the separators and they must not
overlap with any of them.
Next the timing patterns are added, which are two lines, one horizontal and one vertical of alternating
black and white modules. The horizontal timing pattern is placed in the 6th row of the code between the
separators and the vertical timing pattern is placed on the 6th column of the code between the separators.
These patterns always start and end with a black module and may overlap with the alignment patterns
since the white and black modules always coincide.
12
Before the data bits are added, the dark module and the reserved areas for the format and version will
need to be placed as well. The dark module will be placed beside the bottom left finder pattern at the
coordinate (4 ∗ 𝑉 + 9 , 8), while the reserved areas for the format area will be placed one module strip
bellow and to the right of the separator near the top left finder pattern, a one module strip below the
separator of the top right finder pattern, and a one module strip to the right of the separator near the
bottom left finder pattern. The reserved version information area required for codes with version 7 or
higher which are 6 by 3 block will be placed above the bottom left finder pattern and a 3 by 6 block will
be placed to the left of the top right finder pattern.
With that done the data bits can finally be placed. The way this works is starting from the bottom right
of the matrix, we proceed upward in a column that is 2 modules wide, by placing the first bit on the right
side and the second one on its left and then going up a line and repeating this process. Once the top is
reached, we move to the next 2 module column to the left of the previous one and instead continue
downward until the bottom where the upward placement begins again so on and so forth. If a function
pattern or a reserved area is encountered the placement continues on the next unused module.
Fig.10 Data bit placement in a QR code
After all the bits are placed in the matrix, the masking step will commence. Currently there are eight
different mask patterns with different specifications where if the module is masked it will change from
black to white or vice-versa. The mask patterns should only be applied to data modules and error
correction modules.
13
Table 4 Masking patterns for QR codes
Mask
Number
If the formula below is true for a given row/column coordinate, switch the
bit at that coordinate
0
(row + column) mod 2 == 0
1
(row) mod 2 == 0
2
(column) mod 3 == 0
3
(row + column) mod 3 == 0
4
( floor(row / 2) + floor(column / 3) ) mod 2 == 0
5
((row * column) mod 2) + ((row * column) mod 3) == 0
6
( ((row * column) mod 2) + ((row * column) mod 3) ) mod 2 == 0
7
( ((row + column) mod 2) + ((row * column) mod 3) ) mod 2 == 0
When applied to the matrix, each mask pattern has a penalty score based on four rules. The encoder
must test all of the mask patterns and the one which receives the lowest penalty is the one used in the
final output. The way the evaluation is conducted is based on the whole matrix, even though the masking
is only applied to data and error correction modules with the reserved areas considered to be white
modules.
The penalty rules are then as follows:

The first rule gives the QR code a penalty for each group of five or more same-colored modules
in a row (or column).

The second rule gives the QR code a penalty for each 2x2 area of same-colored modules in the
matrix.

The third rule gives the QR code a large penalty if there are patterns that look similar to the
finder patterns.

The fourth rule gives the QR code a penalty if more than half of the modules are dark or light,
with a larger penalty for a larger difference.
For the first rule, if 5 consecutive modules are of the same color, add 3 to the penalty and 1 for each
consecutive one and at the end total the horizontal and vertical results.
14
Regarding the second rule, the person evaluating should look at areas that are 2x2 or larger and add 3
for each possible 2x2 block.
The third rule occurs when a set of black-white-black-black-black-white-black modules followed by 4
white modules on either side occurs that gives a 40 penalty score.
The last rule is based on the ratio of white and dark modules. The total number of the modules in the
matrix counted and the total number of black modules in it as well and the percentage for them is
calculated. For the next step of this rule, the previous and next multiple of five of this percentage must
be found and subtracted by 50 for each of them and take the final absolute value. For the final step of
this rule, each of the previous values are divided by five and then multiplied by the lower of the two by
10 which will give the final penalty score for this rule.
With all of the penalties measured the results for each of the mask patterns can be compared and the
one with the lowest score will be the one chosen to be applied to the QR code.
With the masking pattern done we arrive at the last step of creating the code: the format and version
information strings.
There exists 32 possible format information strings (8 mask patterns times 4 error correction levels) and
it is always 15 bits long. In order to create it, a five bit string is required that encodes the error correction
level and the mask pattern used in the code, which will generate ten error correction bits. These 15 bits
are XORed with the mask pattern 101010000010010 and afterwards the format string will be placed into
the QR code in the areas reserved for them in the arrangement step.
If the version of the code is 7 or larger a version information string with the length of 18 bits will have to
be generated. Six of these bits encode the version while the remaining twelve are error correction bits.
After generating the string the bits are inserted into their designated areas and the final matrix is
obtained.
Table 5 Bits associated with each Error Correction Level
Error
Correction
Level
Bits
Integer
Equivalent
L
01
1
M
00
0
Q
11
3
H
10
2
15
When a QR code is received by a reader, the decoder compensates the geometrical transformation of
the code. The modules are recognized to remove the version information, the information on the used
mask pattern and the error correction level. The QR code is then unmasked to extract the message bits
and the error correction bits. The bits are arranged in blocks depending on the version of the QR code
and its error correction level and are then decoded to restore the original data [8].
Fig. 11 QR code decoding process
2.2 Noise in images
Even though QR codes are mostly known for the resilience to noise interference, it can still happen
when the reader from the decoder detects an image, from the reader being overheated, or the light
conditions in which the image is acquired are not ideal.
One of the possible noises that can affect an image is Gaussian noise. Gaussian noise occurs mostly
in digital images during the acquisition by sensors with poor illumination and/or high temperature and it
will be more dominant in the darker areas of the image [9].
The probability density function for Gaussian Noise is as follows where z is the grey level, µ the mean
value and α the mean value:
𝑝𝐺 (𝑧) =
1
𝛼√2𝜋
𝑒
−
(𝑧−𝜇)2
2𝛼 2
16
Fig. 12 Gaussian noise in a QR code produced in Matlab for variance = 0,1
Salt-and-pepper noise originates mostly from analog-to-digital converter errors or bit errors in
transmission. The image affected by it will have dark pixels in bright regions and bright pixels in dark
regions and it can be mostly eliminated by using dark frame subtraction.
𝑃𝑎 𝑓𝑜𝑟 𝑧 = 𝑎
𝑝𝐼 (𝑧) = {𝑃𝑏 𝑓𝑜𝑟 𝑧 = 𝑏
0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
Fig. 13 Salt-and-pepper noise in a QR code produced in Matlab for noise density = 0,2
Shot noise occurs from variations of the number of photons sensed at an exposure level thus also being
known as photon shot noise. Shot noise follows a Poisson distribution that approximates to a Gaussian
distribution at high levels.
𝑃(𝑋 = 𝑘) =
𝜆𝑘 𝑒 −𝜆
𝑘!
17
Fig. 14 Shot noise in a QR code produced in Matlab
Quantization noise as the name implies is caused by quantizing the pixels of a sensed image to a
number of discrete levels and it has approximately a uniform distribution.
2.3 Color spaces
RGB bases were developed in order to be used as interchangeable spaces to communicate color and/or
as working spaces in imaging applications. These color spaces ease the process of color communication
in the way that a user or device can perceive and understand the color of the image and alter it if
necessary. If applied correctly, the RGB color space will minimize color space conversions in an imaging
workflow, improving image reproducibility and facilitating accountability [10, 11].
Several RGB color spaces were developed as seen in Fig. 15, since there is no consensus on a single
RGB color space that satisfies all imaging needs.
Fig. 15 RGB color spaces
The International Commission on Illumination defined color space as a geometric representation of
colors in space usually of three dimensions as seen in Fig. 16.
18
Fig. 15 retrieved from https://upload.wikimedia.org/wikipedia/commons/thumb/6/60/CIE1931xy_CIERGB.svg/495pxCIE1931xy_CIERGB.svg.png
Fig. 16 Three dimensional representation of RGB color space
The base functions used are color matching functions called CIE floor matching functions. Color spaces
are considered a subset of spectral spaces, which in turn are spaces spanned by a set of spectral basis
functions.
A color flow of a digital image is explained as follows: first an image is captured into a sensor or source
device space. This image may be altered into an unrendered image space (2 standard color space that
describes the original colorimetry) but in most cases it is usually directly transformed from the source
device space into a rendered image space that describes the color space as a real or virtual output [10].
In case that rendered image space describes a virtual output, it is necessary additional transforms to
convert the image into output space that is an output device specific color space.
Most of standard RGB color spaces are under the category of rendered image spaces.
Fig. 17 Digital image color flow [11]
19
When a scene is acquired by a scanner or a digital camera, the first representation of the color space is
device and scene specific depending on the illumination, the sensor and the filters.
When images are archived or communicated in sensor space, the characterization data has to be
maintained so that further color and image processing is possible, and so images should be saved in
TIFF/EP that has defined tags to hold the information.
Depending on the image and the device, the transformation from sensor space to unrendered space
requires linearization, pixel reconstruction, white point selection and a matrix conversion.
The unrendered image color space gives us an estimate of the original colorimetry while maintaining
the relative dynamic range and gamut of the original and if encoded in higher bit-depth will have the
advantage of being processed in tone and color for different purposes and devices with the quality of
the colorimetric estimate depending on the ability to choose the correct scene adopted white point and
transformations.
It is to note that unrendered images need to be placed under additional transforms to make them
viewable or printable and so appearance modeling can be applied when a reproduction is needed.
When color spaces are based on the colorimetry of real or virtual output characteristics they are called
rendered image spaces. These can be obtained by transforming source or unrendered image spaces
as stated before, with the complexity of the transforms varying from video-based approach to image
dependent algorithms [11]. These transforms are mostly non-reversible as some information of the
original scene is lost in the encoding or in the compression to fit the dynamic range and gamut of the
output.
The rendered image spaces and designed to resemble the output device characteristics, to ensure that
minimal loss occurs when converting to the output specific space.
Following the trend of rendered spaces, the transforms to output spaces are device and media specific
as well. If the rendered space is equal or close to the device characteristics, transformations aren’t
necessary, otherwise additional conversions are needed and accomplished by using ICC color
management workflow.
In case the gamut, dynamic range and viewing conditions of the rendered space are very different from
the output space, it is advised to use a reproduction model that allows image specific transforms than
to use the color management system mentioned [10, 11].
20
3. Implementation
3.1 Noise
Now that the basis for the thesis has been laid down, the techniques for spectral multiplexing can be
tested. These techniques can only be applied on same version QR codes since each version has a
specific size and different versions would overlay incorrectly.
The document used as basis for this simulation and further ones along the line in this thesis was “To
Increase Data Capacity of QR Code Using Multiplexing with Color Coding: An example of Embedding
Speech Signal in QR Code” and the website used to transform it into a QR code was
http://www.racoindustries.com/barcodegenerator/2d/qr-code.aspx.
This document follows the word frequency established for the English language very closely, differing
only.in this document for the letter ‘c’ and ‘h’ where they differ for more than 2% in occurrence. Letter
frequency is mostly used as a basis for encoding methods, one of the most commonly known being
Morse code, and as a way to identify languages since each language has a certain distribution attached
to it
Table 6 Letter frequency for the english language
Before starting to experiment on the QR code, an analysis on the effects of noise was conducted on
several versions of QR codes to see the threshold from when a QR code became unreadable.
A sample size of 40 different images, 4 for each version of the QR codes from version 1 to version 10
and the error correction levels. Each of the images was put under test 5 different times for each of the
types of noise and the average of the results was presented in the table below.
21
Table 7 Average values for when the QR code becomes unreadable
Speckle
Code
version
Poisson
(variance)
L
M
Q
H
L
M
Q
H
1
0,2 0,2 0,1 0,1
R
R
R
R
2
0,2 0,2 0,1 0,2
R
R
R
3
0,2 0,2 0,2 0,1
R
R
4
0,2 0,1 0,1 0,2
R
5
0,1 0,2 0,2 0,2
6
Salt & Pepper
Zero mean white gaussian
(noise density)
(variance)
L
M
Q
H
L
M
Q
H
0,2 0,1 0,1 0,2
0,1
0,1
0,1
0,1
R
0,1 0,1 0,2 0,2
0,1
0,1
0,1
0,1
R
R
0,1 0,1 0,2 0,1
0,1
0,1
0,1
0,1
R
R
R
0,2 0,1 0,2 0,2
0,1
0,1
0,1
0,1
R
R
R
R
0,2 0,2 0,2 0,2
0,1
0,1
0,1
0,1
0,2 0,1 0,1 0,1
R
R
R
R
0,1 0,1 0,2 0,1
0,1
0,1
0,1
0,1
7
0,1 0,1 0,1 0,2
R
R
R
R
0,1 0,2 0,1 0,1
0,1
0,1
0,1
0,1
8
0,1 0,1 0,2 0,1
R
R
R
R
0,1 0,1 0,2 0,2
0,1
0,1
0,1
0,1
9
0,1 0,1 0,1 0,1
R
R
R
R
0,1 0,1 0,1 0,1
0,1
0,1
0,1
0,1
10
0,1 0,2 0,2 0,1
R
R
R
R
0,1 0,1 0,2 0,2
0,1
0,1
0,1
0,1
Luckily, Matlab already has tools in place that simulates different types of noise, so it was only necessary
to choose values for variance or in the case of Salt $ Pepper, noise density.
From this experiment it was observable that there isn’t one specific level of noise in which the image is
unreadable since obviously noise doesn’t present itself in the same patterns every time and thus there
is “average” level in which the images do in fact become unreadable, except for Poisson noise where it
didn’t affect any of the QR codes in any of the error correction levels.
Fig. 18 Speckle noise in QR codes for 250 characters for L error correction level and
M error correction with variance 0.04, 0.2 and 0.3
22
Fig. 19 Shot noise (Poisson) in QR codes for 250 characters for L and M error correction levels
Fig. 10 Salt and Pepper noise for 250 characters for an L error correction level with noise density of 0.1,
0.2 and 0.7 respectively
Fig. 21 Zero Mean White Gaussian noise with 0.01 variance for 250 and an L and M error correction
levels for 250 characters
Another thing observed in this experiment was the “deterioration” of the message when being read. In
higher versions the message became more and more deteriorated even if it was still readable in most
cases, often times, characters missing or being replaced by other characters.
23
3.2 Multilevel quick response codes
As stated before, one of methods used to increase the capacity of QR codes was to use multilevel
spectral multiplexing. In this method, different approaches are taken to creating the QR codes such as
choosing different intensities for dark and light modules in the YCbCr color space or choosing module
color based on the its bits using the RGB color space.
Using the first approach, it uses the bases of RGB color by having four levels of intensities which allows
64 different colors to be used, but since the decoder cannot distinguish all of these it is necessary to
choose a set of colors from the available pool [12]. By doing this it was reported that the capacity of the
code increased four times by using 16 colors.
Fig. 22 Multilevel intensities for a grayscale QR code [10]
Fig. 23 Multilevel intensities for color channels in the conventional QR code [10]
The approach in itself is very similar to the approach used for multiplexing three different codes into a
colored one. First two intensity levels for the dark and light modules in each color channel are chosen
for the YCbCr color space in which each value should have a maximum distance between each other
so they can be distinguished from each other, and afterwards they are converted into the RGB color
space.
24
𝑅 = 1.164(𝑌 − 16) + 1.596(𝐶𝑟 − 128)
{𝐺 = 1.164(𝑌 − 16) − 0.391(𝐶𝑏 − 128) − 0.813(𝐶𝑟 − 128)
𝐵 = 1.164(𝑌 − 16) + 2.018(𝐶𝑏 − 128)
Fig. 24 Example of a colored QR code using the method described [10]
When decoding, the RGB color space is converted back to YCbCr color space, separating the color
channels into Y, Cb and Cr and then decoding the bicolor QR code in each color channel [12].
{
𝑌 = 0.257𝑅 + 0.504𝐺 + 0.098𝐵 + 16(𝑌 − 16) + 1.596(𝐶𝑟 − 128)
𝐶𝑏 = −0.148𝑅 − 0.291𝐺 + 0.439𝐵 + 128
𝐶𝑟 = 0.439𝑅 − 0.368𝐺 − 0.071𝐵 + 128
This approach is compatible with current decoders, because when decoding, only the luminance of the
images are used and the chrominance is ignored.
The second approach allows for a standard to be set for multilevel spectral multiplexing. By choosing
the intensity level based on the value of the bit associated with that module and the numbers of QR
codes instead of a random number chosen by each encoder, it will lead to far less errors that might
occur from the first approach and is considered as the best out of two to perform the simulations on.
25
The first part of the simulation was to try and multiplex 2 or more codes into a single color QR code. In
order to do this we divided the red part of RGB into 2 spaces for multiplexing 2 codes. This process
begins by loading each of the images into Matlab using the imread function, giving us a XxYx3 matrix
where the 3 corresponds to the RGB color spaces where 0 is black and 255 is white. After this is done
we chose either the red, green or blue color space as the color we will be working with and transform
the matrix that corresponds to it all to 255. This will make it so that if the color we are working on will be
red (255 0 0), blue (0 0 255) or green (0 255 0) where the black modules were. Afterwards we transform
each of the images to a fraction of their color space in order for them to be added to each other
afterwards. In this case if only the first code was active it would stand at 2/3 of the color, if only the
second one was active it would stand at 1/3, making so that if both were active it would be 0 and if
neither was it would be 1, leading to the images below.
Fig. 25 Examples for colored QR codes using a single color for all the codes
From these images one would say that choosing grey or blue would be the best choice since it’s easier
to ascertain each of the codes, while for green and red some of the lighter areas might be misinterpreted
as white, thus decoding the message with error, but from readers it has been proven that they can in
fact receive part of the correct message.
Now regarding multiplexing more than 2 codes for the same color space we would need to divide the
space in such a way that their sum would lead to 1 but without overlapping colors. One of the ways to
26
accomplish this is by having one of the codes above half of the color space, and the other codes below
it with distinct values as well, so that their total sum would be 1.
𝑥+𝑦+𝑧 =1
0 < 𝑦 < 0,5
0 < 𝑧 < 0,5
𝑥 > 0,5
{
𝑦 ≠𝑧
The problem with this method is that the 2 codes that stand in the less than 0,5 part of the color space
might not be distinguishable between themselves. If let’s say one of the codes is at 0.1 it might be
mistaken as a white module and thus not be accounted for, thus creating a false message.
Another way to go on about this method is by using bits for the amount of codes that we wish to encode.
This is done by first assigning a space in the RGB color space to each code, by using the formula,
2𝑖⁄ 𝑛
(2 − 1), where “n” is the number of codes we have and “i” is a value from 0 to n-1 where 0
corresponds to the first code, 1 the second code and so on and so forth. This method allows for any
number of codes we desire but with each increase, the codes that have a more “light” color start to
become hard to be distinguished among themselves. If the decoder receives the message in its data
form, this won’t pose a problem, but if a picture must be taken first, depending on the quality of the
camera, the message will not be correctly decoded.
Table 8 QR code values for amount of codes being multiplexed
Values for each code being multiplexed
Number of codes being
multiplexed
1
2
1st code 2nd code 3rd code 4th code 5th code 6th code
1
0
0
0
0
0
1/3
2/3
0
0
0
0
3
1/7
2/7
4/7
0
0
0
4
1/15
2/15
4/15
8/15
0
0
5
1/31
2/31
4/31
8/31
16/31
0
6
1/63
2/63
4/63
8/63
16/63
32/63
27
This time, 4 QR codes will be multiplexed. The Matlab process for this is very identical to the one used
before with some minor adjustments. First, each of the QR codes is converted to a selected color space,
in this case, the color space was the blue one.
Fig. 26 QR codes in blue color space
Now that each of the QR codes in in the blue color space, they will be multiplied by a value listed in
Table 8, that is, 1/15 for the first code, 2/15 for the second one, 4/15 for the third and 8/15 for the forth.
The lower the value of the QR code, the “darker” it will be as seen in Fig.27, since the closer the values
are to 0 the closer they will be to the color black.
Fig. 27 QR codes in blue color space after being multiplied by their respective values
The QR codes can now be “added” by performing the imadd command that will add each of the images
to form the final QR code as seen in Fig. 28 on the right
Fig. 28 "Adding” of previous QR codes
28
Regarding the results of this simulation, only the first QR code was “read” by the decoders due to the
way that they are programmed to recognize the darker modules as “black” and the rest as “white.
For smaller QR versions, like versions 1 to 3 it may be advisable to just use higher QR code versions to
transfer data if they are available since the increase in capacity is still surpassed by the higher versions
even after the multiplexing, not to mention the additional time it may take to encode and decode the QR
codes multiplexed.
3.3 Multiplexing with color coding
Another possibility to perform the spectral multiplexing is by using color coding. The way this multiplexing
works is by verifying the values of pixels in the QR code, since there will be a distinct color for each
pattern possible. An example of possible patterns and the respective colors for 3 QR codes is presented
in table 9.
Table 9 Possible color combinations for a color QR code
QR code 1
QR code 2
QR code 3
Index
Red
Green
Blue
Color
0
0
0
0
0
0
0
Black
0
0
1
1
0
0
1
Blue
0
1
0
2
0
1
0
Green
0
1
1
3
1
0
0
Red
1
0
0
4
0
1
1
Cyan
1
0
1
5
1
0
1
Magenta
1
1
0
6
1
1
0
Yellow
1
1
1
7
1
1
1
White
29
Fig. 29 Multiplexing and color coding table for 3 QR codes [3]
If the number of QR codes being multiplexed increases, the color coding table will change and more
distinct colors will be required, such as 16384 colors for 14 QR codes [3]. To generate the color coding
tables becomes then a time consuming and impractical task, but it can be tackled by using MATLAB
tools.
Experiments conducted on this method verified that the data capacity can be increased up to 24 times,
thanks to the 16,777,216 colors available from the RGB color space for 8-bit representation ( 2^24), but
multiplexing over 10 images is not advised since the time to multiplex and demultiplex increases with
each QR code added for multiplexing[3].
Some studies have also considered the usage of symbols instead of colors in the multiplexing and it
follows the same principal of constructing a coding table for each of the possible patterns, but there still
aren’t any major breakthroughs using this method compared to using color so it’s slightly disregarded in
research [13].
30
For the simulation, it was attempted to recreate the multi colored QR code presented. At first a simpler
colored QR code was produced by using only 2 codes for multiplexing. This was done by each of these
codes having a distinct color space and the third space being all white leading to the results in Fig.30.
Fig. 30 Examples of multiplexing 2 QR codes for different color spaces
This simulation could be done as well for any other color chosen as the third color space and isn’t
restricted to white as in the results above.
Fig. 31 QR color code using the first code for red and blue and second code for green
For 3 QR codes, much like in the first simulation, these are loaded unto Matlab but this time, each one
is assigned to a different color, in this case the first code would be red, the second green and the third
blue. The different colored QR codes are then multiplexed like before to create the multi-colored code
in Fig. 32.
31
Fig. 32 Multiplexing of 3 QR codes into one where each code has a different color space
This code, just like in the multilevel multiplexing, has more information in it than a singular QR code of
the same version, and is easier to ascertain where each of the 3 codes are, even when using a camera,
but the current QR code readers do not have the ability to do so and are unable to even form part of the
message like it did when it was used for multilevel where at least the first code was read.
The problem for this method would also be that in order to multiplex more than three codes, there would
have to be a particular type of color for the extra codes and these could not be the combination of any
of the 3 previous codes, nor could they originate another color that already exists and is being used
when added to another code like purple or yellow.
32
3.4 Multilevel color spectral multiplexing
A simple solution to the problem for the color coding would then be to merge the 2 methods presented
in this section, that is, to use the same color but “lighter” for the new codes, thus while the color
technically would be the same, the different “aspect” of the color would allow to circumvent the problem
and allow for multiplexing of more than just 3 codes.
The way this is implemented in Matlab is to use the bit equation posted above for each pair of QR codes
(1-2, 3-4, 5-6 for example in the case of multiplexing 6 QR codes) where each pair has a different color
assigned to them and each of the codes in that pair has a different tone of that color assigned by the bit
equation. Afterwards the codes are multiplexed together just like in the previous section originating the
QR code seen below.
Fig. 33 Example of multiplexing 6, 9 and 12 different codes into one
From the figure above it is possible to observe that this last method is probably the best out of all the
listed, since each of the codes are clearly distinguishable even if they have a “lighter” color than some
of the other ones, unlike if only one color scheme was used, also unlike the color QR code method there
33
is no need to specify any other color besides red, green and blue since all of the codes will be one these
three and the multilevel part of the encoding will make sure that when multiplexed, no color will be
repeated.
It is of note as well that while in this simulation the QR codes were paired in a simple way so that it was
easier to decode, there is no need for this to happen in further development of this method, since if the
person encoding so chooses, they can put all the codes except one in the same color space and this
method would still work.
This method only requires that either a standard is set in further development for the order of the
decoding to occur, and for a means to transmit the information of how many QR codes are in that color
space to hasten the decoding, although this last part can be skipped at the cost of spending a couple of
seconds analyzing the values in the matrix and the program figuring out for itself based on them, how
many QR codes are on the color space.
For the final part of this simulation, a Matlab program was created in order to de-multiplex and decode
the final message transmitted in this type of code, which performed the following tasks in order to decode
it:
1. Separate the codes from the original multiplexed one by separating the color spaces and
observing how many different values exist and performing log 2 𝑛 where n is the total of different
values;
2. Obtain the EC level, mask pattern and version of the code;
3. Substitute values that won’t be important to the decoding of the message so it won’t interfere
with the decoding, such as the finder, alignment and timing patterns and the dark module and
seperators;
4. Remove the masking pattern applied on the code;
5. From the number of total bits, perform a for cycle that will read the code and save the bits in an
array;
6. Convert array from bits to characters to obtain decoded message.
Since each of the bits had a specific value assigned to them, and the image was received from one
previously created and not acquired through a camera, the message did not require the error correction
bits to be considered since no damage had occurred and was decoded correctly.
4 Large capacity QR Code
34
4.1 Image transfer
Now that it was shown that a larger message can indeed be encoded in QR codes through multiplexing,
let’s take a closer look at the data capacity and the time it takes to multiplex and demultiplex these
codes.
In a normal QR code, about half of its bits are error correction bits so that in case the QR code gets
damaged, the message can still be received. Although the error correction bits are obviously important,
it is also space that could be used to include more information, if the QR code would not be damaged
in any way.
Let’s consider then that the QR code would be transferred in a digital way that would not get damaged
then: each code has a RGB part to it, this means that their theoretical total capacity would not be,
((𝑣 − 1) ∗ 4 + 21) 2 ,where v is the version of the code, but instead three times that, so while a version
1 QR code has 441 bits if purely seen by the first equation, it actually has the ability to hold 1323 bits if
we make use of the three RGB color spaces.
But although this is the highest amount of bits a version 1 QR code could hold, it still needs its finder
patterns, alignment patterns, separators and timing patterns in order to be a recognizable QR code,
which will reduce the amount of bits by around 606 bits of the 1323 estimated total in this case.
Considering that the total amount of bits for a byte mode in a version 1 code is 136, by using the RGB
color spaces and disregarding the error correction bits, the amount of data for this version would
increase by 5.5 times.
Of course this is only considering that only the color multiplexing could be used. As it was proven,
multilevel also works to increase the capacity of the QR code while working in conjunction with color
multiplexing, so the capacity of the QR code could become far greater than just by using color
multiplexing.
The only limitation to this is the time that it takes to perform the multiplexing and demultiplexing of the
QR codes. As was stated in the section 3.3, by multiplexing and demultiplexing 10 QR codes and and
performing the decoding, it took over 20 mins for the task to be completed, which is too much time to
decode a QR code that is known for its quick response, but once again this is considering that the error
correction and masking occurs for each of the codes so that the message is decoded correctly.
If the QR codes can actually hold such high amount of bits, they could potentially hold and transfer the
data of other things such as images or even sound, if their data can be converted to bits or are already
in bit form.
A simulation was performed for both these cases, to see how much time it would take to encode/decode
each and if it was possible to fully recreate the original piece.
35
In the early stages of the implementation, a 50x50 JPEG was chosen since its size would “fit” into several
of the large QR codes simply by putting the RGB values inside, without the need for them to be converted
into binary.
For the first step in this stage, the smallest version QR code possible was found on the basis that the
total number of bits hard to be larger than the width times the height of the image being inserted, which
in this case was 2500 bits long at least.
Since at this stage the focus was more on the amount of data transferred, the alignment patterns were
ignored and only the location and timing patterns were included.
The image created included a bit representation at the start that would identify the height and width of
the original image in order for it to be recreated, followed by the RGB values of the image being tested
as shown in fig.34.
Fig. 34 50x50 image and QR code holding its information
The image was in fact decoded correctly using this method of inserting its RGB values into the QR code,
but the problem arose for bigger types of images such as 403 by 530, that did not fit inside of a version
40 QR code.
Multilevel was then considered as a way to tackle this problem since it was proven to be an efficient way
to increase the capacity a QR code, but it carried a new problem with it, since that while before when
using multilevel the only values multiplied were 0 and 255 to be used in the multilevel QR codes, now
there were all the ones in-between those two values and there was not an easy way or even fail proof
way to recover them once multiplied since the program would narrow each value after the multiplication
had taken place, so a new approach had to be taken.
The best approach was then to use the same way normal message were encoded and that was by
transforming the data into bits. Since all the values in the image go from 0 to 255, an 8 bit string for each
of these values was enough, but that obviously meant that the data to be inserted into the QR had to be
increased by 8 times
36
A version 18 QR code was then chosen to be multiplexed by 9 different QR codes, 3 for each of the
color spaces since it could hold a total of 23763 bits for the images’ required 20000 bits, but a smaller
code can be chosen if need be, by increasing the amount of QR codes to be multiplexed.
The alignment patterns were also included to be able to distinguish each version of QR codes since it
would also help with possible image distortions.
After the size of the data and the reserved areas were placed, multilevel was applied to each of the RGB
color spaces, and the QR codes multiplexed together into fig. 35.
Fig. 35 Version 18 QR code holding image information
For the decoding, like for the multilevel method, the reserved areas are altered in order for the cycle to
ignore them when scanning the QR code. After receiving the dimensions for the image, the code is
scanned X amount of times until the total amount of bits received by the images’ dimensions is reached.
With each iteration of the scanning, it considers a different QR code is ‘active’, as in for the first iteration,
the first QR code is considered ‘active’, and searches for the values in the matrix that corresponds to
the possible combinations for that QR code for bit 0 and bit 1 as seen in table 10.
Table 10 RGB values for 3 QR codes being multiplexed
RGB Values
Bits
37
255
108
180
36
216
72
144
0
1st code 2nd code 3rd code
0
0
0
0
0
1
0
1
0
0
1
1
1
0
0
1
0
1
1
1
0
1
1
1
As seen in fig. 35, there is a “lighter” part of the code on the left. This is due to the third QR code not
being fully filled with data since it was not necessary, and so the left over bits were all considered as 0.
The encoding ended up taking 10 seconds to multiplex all 9 different QR codes and less than 3 seconds
to demultiplex and reconstruct the original image.
From this simulation it is possible to conclude that indeed this type of data can be transferred in QR
codes using the method above although for this image, the bytes were less than the bytes of the image
produced. This can possibly once again be circumvented by using a lesser version QR code and
increasing the amount of QR codes multiplexed, thus reducing the byte size of the image at the cost of
a slightly higher encoding and decoding time.
For the 403x530 JPEG previously mentioned it was noted that it required over 162 QR codes being
multiplexed for a version 40 QR code (54 for each RGB color space). This would lead to an enormous
time for encoding and especially decoding the QR code to recreate the image.
If the aim of the person encoding is simply to transfer the data without worrying for the
encoding/decoding times, this method can be used for an image of that size, since the byte size of the
QR code will be less than that of the image being encoded.
38
4.2 Audio transfer
Regarding the sound file simulation, a MP3 file with the duration of 1.5 seconds and 23 Kb was chosen
for this simulation.
Like in the image simulation, the first step is to figure out how much data is going to be transferred. This
was performed with Matlabs’ audioread function which gave us 131318 entries (65659 by 2), since the
audio file was in stereo. For the simulation only the first column was considered for the encoding.
The task now was to figure out how to transfer said audio data into the QR code. Unlike the image, the
audio file didn’t possess values that would translate easily into bit form, since they were comprised from
-1 to 1 and some of its values went as low as 10−6 .
The way this was tackled was by converting these values into more manageable ones. For negative
values, they were multiplied by -150, thus allowing them to be converted into bit form and making it so
that lower values could be “seen” as well. For the positive numbers, they were multiplied by 150 and
added 150 afterwards.
This allowed for the data to be encoded as 8 bits, but for the lower values this was still not enough to be
able to be distinguished and so they were considered simply as 0 for the remainder of the simulation.
The 40 QR code version was chosen, since it is the largest QR code available, and while it is possible
to encode the data with smaller QR codes, this version needs 18 different codes to be multiplexed in
order for the whole data to be transferred, which means that for lower versions, like 30, 20 and 10, the
amount of codes necessary would increase by 1.7, 3.4 and 10.8 times respectively.
The creation of the QR code is basically the same as the one described for the image coding. We begin
by first reserving the areas for the several patterns to be inserted into each of the 18 codes and then
start to insert the audio data itself.
The first information regarding the data inserted is its total size, just like it was for the image, so that
the original audio can be recreated. After this is done, the data for each entry can be inserted in its bit
form, performing several ‘for’ cycles to insert the data into the 18 different QR codes.
Unlike the image simulation, the audio does not have an RGB color space so there is no need to
specifically insert the bits in different color spaces like in the image simulation where it helped the
decoder to have it be like that.
For the audio file, the data can be inserted in anyway without needing to be ‘restricted’ to the color
spaces, meaning that if the person encoding wanted to use all 18 codes in the red space, they could
do so, but for an easier way it was considered that it was better to divide the codes through all 3 color
spaces, since 18 is divisible by 3. If the number of codes was not divisible by 3, then one or two color
spaces would have extra QR codes assigned to it.
39
After the data was inserted into the QR codes, the patterns were added and each pair of 6 QR codes in
each color space is converted so that they can be multiplexed like in the image simulation.
For the decoding of the QR code, first each color space is separated, the amount of data is acquired
and the pattern areas are altered to a value that won’t interfere with the reading of the QR code.
For the retrieval of the data inserted in all 18 QR codes, 6 ‘for’ cycles are performed for each of the color
spaces to retrieve the bit values for the QR code associated with that cycle, as seen in the image
simulations’ table 10.
The initial results yielded a lot of noise due to problems with smaller values having the same ones as
the negative ones, such as 0,00005 ∗ 150 + 150 = −1 ∗ (−150), since in Matlab the values are rounded
up. The way this was solved was by considering that values with less than 2 decimal points were ignored,
thus reducing the noise that occurred.
Another problem that arose in the decoding was the fact that a column was repeated in column 92,
which lead to an initial error while decoding. There is no easy way to deal with this since it may be
possible that the repeated column may not be a repetition and in fact a correct column. The column was
possible to be identified since it was next to an alignment pattern and it made the alignment pattern into
a 5x6 instead of a 5x5.
The encoding for this audio file took a minute to complete while the decoding took almost 2 minutes to
demultiplex it and reconstruct the audio file.
Fig. 36 Version 40 QR code holding data file information
40
1
0.8
0.6
Amplitude (au)
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
-1
0
0.5
1
1.5
1
1.5
Time (s)
1
0.8
0.6
Amplitude (au)
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
-1
0
0.5
Time (s)
Fig. 37 Audio file and reconstructed audio file
Although it is possible to distinguish the differences between both versions, the decoded version was
audible and presented no noise even if the values weren’t identical to the original.
41
The result was also put under SNR and MSE analysis, for the original and the reconstructed audio file,
by applying their respective formulas bellow, where x(n) is the original audio file and y(h) the
reconstructed audio file. The estimated values for SNR is 36,03 dB and 0.0000134 for the MSE.
𝑆𝑁𝑅 = 10 log10 (
𝑀𝑆𝐸 =
∑𝑛 𝑥(𝑛)2
)
∑𝑛(𝑥(𝑛) − 𝑦(𝑛))2
∑𝑛(𝑥(𝑛) − 𝑦(𝑛))2
𝑛
The ability to ‘insert’ audio files may be a useful way for advertisement companies or the music
companies to distribute small audio files like ring tones, without the consumer needing to download the
file from the internet.
42
5. Conclusions
From the results obtained in chapter 3 of the thesis, it is possible to verify that using spectral multiplexing
on QR codes is a viable method to increase their capacity by at least 24 times. By using this method, a
smaller version code can transmit the same amount of data that would take to be transferred in a higher
version code, while maintaining the amount of bytes that the smaller version has. This would allow for a
possibly more economic approach for companies to divulge their products or for hospitals to add more
information about their patients to their bracelets.
The possibility of audio being transmitted in a QR code by using spectral multiplexing could open up a
new venue for the music industry and the advertisement industry to divulge their works by creating QR
codes that have audio files in them, such as ringtones, so that when someone scans the code, that
person could decode the QR code and listen to the audio file with almost no loss in quality, without
requiring an internet connection.
Since there already exists some types of colored 2D codes, the code used for the readers could maybe
be applied to the ones that decode QR codes, else the way one would go about changing the way the
code works for the QR code readers, would be to separate the image into its RGB color space, verify
how many QR codes were multiplexed by using the formula, 𝑡 = 2𝑛 , where t is the different values
presented in the color space and n is the number of QR codes that were multiplexed, and separate
these values into each of their original QR codes and then read them as normally.
The only limitations this method has are the printing of QR codes since different devices from different
manufacturers reproduce the RGB color space in different ways, the dependency on camera phones
for capturing the image in the most adequate illumination and brightness and the amount of time that it
may take to decode a multiplexed QR code with a high amount of QR codes in it, but this can be
circumvented by some form of code optimization and a faster processor, along with if the means of the
data transmission is purely digital, the camera and printing problems can be disregarded.
43
44
References
1. Max E. Vizcarra Melgar, Alexandre Zaghetto, Bruno Macchiavello, Anderson C. A. Nascimento,
“CQR CODES: COLORED QUICK-RESPONSE CODES”, 2012 IEEE Second International
Conference on Consumer Electronics - Berlin (ICCE-Berlin), 2012
2. Hiroko Kato,Keng T. Tan,Douglas Chai, “Barcodes for Mobile Devices”, Cambridge University
Press, 08/04/2010
3. Hiren J. Galiyawala and Kinjal H. Pandya, “To Increase Data Capacity of QR Code Using
Multiplexing with Color Coding: An example of Embedding Speech Signal in QR Code”, 2014
Annual IEEE India Conference (INDICON), 2014
4. Denso ADC, “QR code Essentials”, 2011
5. Jose Joaquin Mira, Mercedes Guilabert, Irene Carrillo, Cesar Fernandez, Ma Asuncion Vicente,
Domingo Orozco-Beltran, Vicente F. Gil-Guillen, “Use of QR and EAN-13 codes by older
patients taking multiple medications for a safer use of medication”, INTERNATIONAL
JOURNAL OF MEDICAL INFORMATICS, 2015
6. Yanghua Gao, Zhihua Zhang, Huanwen Wang, Hailiang LU, “Two-Dimensional Code Based
Spatiotemporal Modeling and Its Application in Object Tracing”, INTERNATIONAL JOURNAL
OF DISTRIBUTED SENSOR NETWORKS, 2015
7. Artur Carnicer, Amir Hassanfiroozi, Pedro Latorre-Carmona, Yi-pai Huang, Bahram Javidi,
“Security authentication using phase-encoded nanoparticle structures and polarized light”,
OPTICS LETTERS, JAN 15 2015
8. Hou A-Lin, Feng Yuan, Geng Ying, “QR code image detection using run-length coding”, 2011
International Conference on Computer Science and Network Technology, 2011
9. Dr. Philippe Cattin, "Image Restoration: Introduction to Signal and Image Processing". MIAC,
University of Basel, 2012
10. Danny Pascale, “A Review of RGB Color Spaces … from xyY to R’G’B’”, The BabelColor
Company, 2002-2003
11. Sabine Süsstrunk, Robert Buckley, Steve Swen, “Standard RGB Color Spaces”
12. Masanori KIKUCHI, Masaaki FUJIYOSHI, and Hitoshi KIYA , “A New Color QR Code Forward
Compatible with the Standard QR Code Decoder, Intelligent Signal Processing and
Communications Systems (ISPACS), 2013
45
13. Sartid Vongpradhip, “Use Multiplexing to Increase Information in QR Code”, The 8th
International Conference on Computer Science & Education (ICCSE 2013), April 26-28, 2013
46