Pool Application
Sharon Sultan – 039779228
[email protected]
Gilad Levy – 204421945
[email protected]
1.
Abstract
Pool is a board game, which includes 2 players, a board and
cue balls. Each player must put all the cue balls belonging to
his color, into the holes in the field.
Pool-Application is Android based application which offers the
players all the possible moves on the board and their difficulty
level.
In this report we’ll discuss the methods we used in order to
achieve our results, the features and the limitations of our
application.
2.
Introduction
We’ll discuss the three parts of our application.
(1) Client side, which is the GUI of the application, locates on
the user's android mobile instrument, and responsible to send
the board image to the server. The client side also display the
given moves after analyzed by the server.
(2) Vision part happens in the server side, and written in
matlab. Responsible to process the image, extract board
measurements and the cue ball's colors and positions in the
board.
(3) Finally, the ‘brain’ of the application, written in java,
calculates the possible moves and rank their difficulty level by
self-written algorithm. The algorithm takes into account
distances and angles and after analyzing, it sends the
information back to the client which displays the results.
The goals of the project was: (1) 100% board and cue balls
detection, (2) calculation of possible move and their difficulty
level and (3) robustness to angle and illumination changes.
3.
Graphical User Interface
The graphic user interface consists of three screens, which
enable the user to configure, take and calculate new board state
and browse the output, possible suggested moves.
(1) Welcome or the first screen is a camera preview with two
buttons, one takes the user to the configuration screen which
we shall mention later, and the second will grab the picture
presented to the user and send it for processing in the server
side. After the picture is taken, it is being converted to JPEG
format with quality parameter of 0.75, what reduce the size of it
from > 3M to about ~500Kb. The picture is then being sent to
the server using restful multipart HTTP request. When output
from the processing is received back, it is being passed to the
next screen, the shots browser.
(2) The shots browser screen, received the output of the
processing of the board, which is basically two lists of all
possible suggested shots, one for each player. On this screen,
the user can navigate between shots. Shot is presented on
screen, as set of lines that represent the predicted or optimal
path for that shot, such that, if the user hits the ball correctly, it
will probably enter the desired hole. Navigation panel in located
on the bottom on the page. For every shot, there is a difficulty
indicator in the top panel, the indicator tries to provide the user
with a rough sense of how difficult is it going to be to take the
shot as shown on screen. There are four levels of difficulty
(easiest to hardest): (a) Piece of Cake, (b) Fair Shot, (c) Hard One,
(d) Expert Shot. For simple shot, the difficulty is calculated based
on the distance which both of the balls have to travel (target ball
and hitting ball), as well as the angle of which the target ball is
being stroke. For complex shot (one that hit the bank), the level
is Expert by default.
(3) Configuration screen, in the screen the user can configure
the address and port of the analysis server.
Shots Browser Screen
Example of an input image
4.
Computer Vision Methodology
Pool Application mains concept was locating the board, the cue balls
and its colors, from vast range of angles and illumination changes.
Computer vision has been implemented on server side, written in
matlab. We’ve divided the procedure to three main parts, board
alignment, locate all cue balls and their colors and transfer the data.
The data includes board measurements, cue balls and the projective
transformation matrix.
We’ll briefly discuss about each step:
(1) Isolate the board – With the help of the RGB
channels, we’ve located independently the
green part of the image. Which allowed us,
using clustering, to label each pixel as one or
zero, due it’s green value.
(2) The next challenge was locating the board
corners. That challenge was important to
overcome as projective transformation
requires at least four points. We produced an
edge image (using canny algorithm) and used Hough lines in
order to find four peaks of Hough space. Solving the four
equations gave us the corners of the board. In addition, we
have implemented an exception for really low image angles,
to avoid inaccurate results. Our algorithm searches for fifth
intersection between the lines within the borders of the
image. If found, the application displays a request on the
android’s screen to take a picture from higher point of view.
(3) Given our points of interest we use POI function to blank out
all pixels that not in our interest, leaving only the board. This
step is important for two main reasons, all cue balls that will
be found must be located on the board, which will eliminate
noise. In addition, to determine the true color of the cue
balls we used normalized channels, which important to
perform this without taking any noise into accounting.
(4) Normalize the RGB channels of the board. In order to do so,
and taking account only the pixels within the borders of the
board, we perform on each channel: R = R / sqrt(R^2 + G^2 +
B^2). We've succeeded to reduce magnificently the impact of
different illumination on the board.
(5) Using Hough circles on the
normalized image provides all
cue balls positions.
Imfindcircles method in matlab
produces centers of circles and their radius for given radius
range as input. Because Pool Application supports vast range
of angles, the radius range has to support different sizes of
cue balls as well. As close cue balls 'seem' larger and far cue
balls 'seem' smaller, we determined radius range as 15-45
pixels.
(6) Transform normalized
image to LAB color space.
LAB color space project
the 'true' color of the cue
balls. As an example, RGB channels do not separate well
enough between red and yellow cue balls.
(7) For each circle found, Pool application determine its color by
median of ‘a’s and ‘b’s of its neighbors. We choose the scale
of the pixel neighbors by the radius found through Hough
circles.
(8) Finally with all cue balls and its color have found, we use the
previous calculated projection matrix and transform the
board measurements and cue ball position to known
coordinates space. All data is written to text file which later
will be processed.
5.
Pool model calculation
Pool model calculation is the second phase of analysis that being
perform in the analysis server. When an analysis request is received
from that client, the web server saves the received image to the disk,
and initiate a call to the matlab code that perform the vision analysis
described in section 4. The output of the vision analysis is a text file,
which contains the necessary data for solving the current state of the
board as it presented by the received picture. The text files contains (1)
measurements of the board. (2) All cue balls positions.
(3) Transformation matrix to the original image coordinates. The
server parses the result file into model objects. With the model objects
in hand, the server iterates for each player over its ball on board, and
for each ball it calculate a possible shot for every hole. First a simple
shot is being tried, if no simple shot exists, a complex shot will be
calculated if possible.
In order to calculate the possible shots server uses simple geometric
rules in order to have the hitting ball hitting the target ball in the
desired hitting location from the correct direction.
After a shot was calculates server checks whether it is feasible, it does
so by verifying that the distance of every ball on the board is greater
that the diameter of a ball.
If a player has no balls on the table, server will calculate for him shot
with the black ball as the target ball.
A shot object is actually a set of lines, before the data is ready to be
sent back to the user, the server transforms back, using the provided
projective transformation matrix, all the lines back to the original
image coordinates. The output, is ready at this point to be returned to
the client application that will draw the shots on the original image
using the shots browser described above.
6.
Limitations
(1) Our first and main limitation was that instead of using regular billiard
balls (all colors, solid and striped) we used only solid snooker balls with
two colors, red, and yellow. One for each player.
(2) We calculate two kinds of shots (a) Simple shots, a shot in which the
white ball hits the target ball directly and target ball goes directly to the
desired hole (b) complex shot of type 1, a shot in which the white ball hits
the bank before hitting the target ball. We do not calculate complex shots
of type 2, where the white ball hits the target ball directly, target ball hits
the banks and enters the hole. More complex shots are also not
supported.
(3) In some angles (very low) the server fails to analyze the board, so we
use a threshold over the angle in order to avoid a bad analysis of the
board. The angle is very low, so basically user won’t encounter the
message we issue in that case
(4) We assume a reasonable illumination, even though the color
detection mechanism is extremely robust, player may still encounter
mistakes in color or wrong positions for balls if illumination is very poor.
7.
Results
We'll discuss Pool application results in several fields, board processing
from different angles and illumination directions, cue balls color
detection accuracy.
(1) Pictures taken from angles of -+ 60 degrees from the board and up to
3 meters are accurate for 95%, Meaning 1 of 20 images will be meshed
up.
(2) Using the flashlight helped to improve the robustness to different
illumination directions and colors, so except images with no external light
source, different illumination have no effect on the results.
(3) 1 of 15 images has inaccurate cue ball model. Which can be a missing
cue ball, additional cue ball or un accurate color.
The results are calculated in the server side and using the projection
matrix, they transformed and sent to the android device. The taken image
displayed on the android and a UI allow to through all calculated shots.
Shots are displayed as lines which represent and shot between the white
ball and the target ball, including the exact position to hit on the target
ball, and the line between the target balls to the hole. Based on the
calculation on the server, a difficulty level is displayed for each shot.
Pool Application is working fast, accurate and has robustness to angles
and illumination changes.
8.
References
(1) Find Circles using circular Hough transformation
http://www.mathworks.com/help/images/ref/imfindcircle
s.html
(2) Using LAB color space
http://www.mathworks.com/matlabcentral/fileexchange/
24009-rgb2lab/content/RGB2Lab.m
(3) Billiard shots calculation
http://www.easypooltutor.com/articles/59-bank-and-kickshots/154-intro-to-bank-shots.html
http://www.quora.com/How-do-you-calculate-what-angleto-hit-a-pool-ball
(4) Math works - matlab methods examples
http://www.mathworks.com/