MBrowser v1.7 (Multimedia Browser)
Quick Reference
by S. KIRANYAZ
MBrowser is the primary and only MUVIS application that is used for multimedia browsing and
retrieval within a MUVIS database. It is a powerful multimedia terminal, which furthermore
supports database visual/aural rendering, retrieval via query by example, Progressive Query, video
summarisation, etc..
1 Installation
Simply run “Setup MBrowser v1.7.exe” file. Once installed the same file can be used to uninstall
the program.
Q: Which operating systems MBrowser can be installed?
A: Like the other MUVIS applications, it is developed and tested for the following OSs:
• Windows 9x
• Windows ME
• Windows 2000
• Windows XP
Q: Are there any pre-requisite programs that should be installed before MBrowser
installation?
A: Yes. It is strongly recommended to install MPEG-4 codecs such as DivX and XviD.
http://www.xvid.org/ & http://www.divx.com/
2 MBrowser in Run Time
When you run, MBrowser UI appears on the screen with no current action. User should initiate the
actions.
Q: What are the actions that MBrowser can perform?
A: In basic terms, MUVIS multimedia (MM) items such as audio/video clips, MUVIS databases,
images and frames are rendered and used for retrieval using MBrowser. You can load a MM item
simply by drag-n drop into the main UI or you can use <File> menu to load them.
Q: What types of databases MBrowser supports?
A: Any type of MUVIS databases with such extensions: vdbs, hdbs & idbs.
Q: What happens when I load a database?
A: According to the rendering mode, 12 MM items of the database that are randomly chosen within
the database are displayed on the left side of the UI. If the database is a hybrid database, the 12
Audio/Video clips are initially displayed.
Q: How can I change the Hybrid database display mode from video to images or vice versa?
A: From <Display Functions> Æ <Hybrid Database Options> sub-menu.
Q: How can play/view a database MM item?
A: Just left-click on it OR chose playback/clip from the right click menu.
Q: How can I query a database MM item?
A: There are 2 ways of initiating a Query Process; the first one is to start playback or view it as
explained and then you can press Query button to query it immediately (if the database is indexed
by some features). The second way is to use the right-click menu to start query immediately.
Q: If I stop playback or pause playback or choose another item from the database, is Query
affected?
A: No. Essentially it is recommended to stop or pause playback if you query a video/audio file to
give more CPU power to Query process. But there is no harm to do anything else the query will be
performed anyway.
Q: I loaded a database and it gives a Warning message saying that the database contains no
features and therefore, the query is disabled. Why?
A: At least (theoretically) one MM item in the database should have one feature vector extracted in
order to establish a query. In other words the database should be indexed by some feature vectors in
order to enable the query process. If you query a video or audio clip then that clip should have at
least one feature indexed along with the clip.
Q: How can I see whether a database does contain features and the type of the features?
A: There are various ways of checking the database features: Load a database into MBrowser and
from <Query> Æ <General Query Parameters..> menu you can see the visual (if exists) and audio
(if exists) features indexed into the database. The second way is to load the database file by a text
editor and check for the features all of which either “visual” or “audio” tag preceding.
Q: How many Query modes are supported by MBrowser?
A: This basically depends on the types of the features and MM items existed within the MUVIS
database. Once a database is loaded into Mbrowser, from <Query> Æ <General Query
Parameters..> menu you can see the visual (if exists) and audio (if exists) features indexed into the
database. So if visual feature(s) are present, you can perform “Visual Query” and similarly if Aural
feature(s) are present, you can perform “Audio Query”. Visual queries can be performed over video
clips, key-frames and images. However audio queries can only be performed over audio clips (with
or without video).
Q: How can I change the query mode between Visual & Aural Queries?
A: First choose <General Query Parameters..> menu to get Query Parameters dialog. In this dialog
there is a switch button at the top saying “Visual Query is Enabled”. You can change the Query
mode from this switch.
Q: How can I change query parameters?
A: As explained above, choose <General Query Parameters..> menu to get Query Parameters dialog.
In this dialog you can enable/disable the features used for query and you can give weights for each
sub-feature (if enabled) using the slider bar or typing the number (<1.0) into the edit box. The
similarity distance obtained from that sub-feature vectors will then be multiplied by that weight
number to calculate total similarity distance.
Q: What is Progressive Query (PQ)?
A: PQ in one of the important features from MUVIS v1.6. It is the default Query type in MBrowser
(so you do not need to change anything) and it enables the user to get the progressive query results
periodically. You can set any period from Query Parameters dialog but make sure that to enable
Progressive Query beforehand. If you do not enable it then the query type will be Normal Query
where the results (the final) will arrive after the query process is completed.
Q: Can I stop or browse through the Progressive Query retrieval results?
A: Yes. By turning the Query button in the main window of MBrowser one can browse among the
query retrieval results. Also you can use directional keys (up & down) for this. In order to stop an
ongoing PQ simply press STOP button (same as Query button). It will stop as soon as next periodic
sub-query is completed.
Q: What is HCT Browsing?
A: HCT stands for Hierarchical Cellular Tree. It is a novel indexing structure that is integrated into
MUVIS v1.7 recently. If a MUVIS database is indexed by HCT then there are two potential use of it:
within PQ for faster retrievals and for HCT Browsing. PQ over HCT is the default query method it
is automatically used when a query is performed. HCT Browsing on the other hand is the alternative
browsing method for random and sequential browsing.
Q: How can I do HCT Browsing?
A: Choose <Display Functions> Æ <Database Browsing Options> Æ <HCT Browsing>. Similar to
query types there are two different HCT Browsing modes available: Visual and aural, both of which
uses the appropriate HCT indexing structure –if present. If not, the query browsing will
automatically be set as the default browsing mode (random browsing). When the HCT Browsing is
selected then the HCT Browsing Control Window will pop up and you can browse through the
HCT body using this window.
Q: How can I use HCT Browsing Window?
A: As shown in Figure 1 the HCT Browsing Control Window provides buttons to navigate among
levels and cells within the same level. The hierarchical structure of HCT can be used to give an
overview to the user about what lies under the current level so that if well supported via a user
friendly GUI, HCT Browsing can turn out to be a guided tour of the database items. When the user
initiates it, it is designed to show the items in the cell at the top level, so it is the first clue about
what the database content or more specifically a brief summary of it. The next logical step for you
is to choose an interesting item in this cell and starts the tour downwards. So this is the first
functionality that HCT Browsing provides: to choose an item in the upper level and trace down to
see the cell it represents (as a nucleus item of that cell). As long as the item chosen belongs to the
current cell, the “level down” option will lead to the lower-level cell that is represented by that item.
Otherwise, the first cell in the lower level will be shown by default. In the ground level the “level
down” option is naturally disabled. The opposite functionality, the “level up” will yield the upper
level cell, which is the owner (cell) of the nucleus item of the host cell at the current level and it
works at all levels except the top level. This is also a useful functionality to browse similar cells of a
particular cell. Therefore, the (slight) variations of a particular content can be visited using both of
the level functionalities.
In addition to such inter-level navigation options, HCT Browsing provides inter-cellular trips
within a certain level. You can visit the cells sequentially, in a forward or backward direction and
one cell at a time. This is especially useful when you do not have any particular target content in
mind and may be just “looking” for interesting items. So in such an indefinite or “open-ended”
exploration task, navigating through consecutive cells at a certain level will summarize the overall
database content and the amount of summarization obviously depends on the “height” of the
navigation performed, that is, simply the current level number.
Nucleus
Item
Level
Controls
Cell
Controls
Cell
Items
HCT
Info
Item
Navigator
Buttons
Cell MST
Info
Prev Next
Figure 1: HCT Browsing Contol Window
Q: When I query an image/key-frame, I saw a curve in the left side, what is it for?
A: The features of images are displayed in graphical form for comparison. The upper black graphic
area is for the queried image features. The lower black graphical area can be used to sketch the
feature vector of any image on the database side. Just right click menu on an image and choose
<Show Features...>. You can switch to any other feature by choosing an available feature from the
list that is under the graphical displays.
Q: Is graphical feature display for only images?
A: Yes. Since it is not suitable for video and audio, each of which have multiple frames and so
multiple features.
Q: Once the query is finished, what exactly is displayed on the UI screen?
A: The descending order of query results with the similarity distances reported under each MM item.
So the best match is the upper left one. The second best match is the upper middle one and so on..
Q: There are only 12 best matches displayed then?
A: No. Once a query is performed, then the arrow buttons on the right down corner of the UI screen
becomes active. You can see the 2nd best 12 results, 3rd best 12 results, etc. by pressing “Next”
button.
Q: How do I know which button is used for what?
A: Bring the cursor on top of them and you will see pop-up of a data tip reporting the button
function. The same is true for the MM items of the database.
Q: Sometimes I am loosing refresh on the UI screen, what should I do to refresh the screen?
A: You can manually refresh the screen simply by clicking anywhere in the UI window.
Q: How can I do video summarisation?
A: There are several ways of doing it:
• If you want to see only the Key-Frames (KFs) of the video clip, you can activate the KF
animation of a video clip by pressing right-click menu and choosing the <KF Animation>.
You can then use stop button to stop it or pause button to pause it. If you pause it you can
KF next and KF prev buttons to see the KFs manually.
• If you want to see KFs with the video same time, just right-click menu and choose
<Playback with KFs>.
• If you want to see KFs all in once, then from main menu, choose <View> Æ <KFs & SFs
Window>. A new dialog called “Key-Frames Display Dialog and Scene Detection” will
emerge. Load a clip and you will get all the KFs shown in the UI window. You can also do
Scene Detection to extract to Scene Frames (SFs) out of KFs.
Recommendation: KFs & SFs window may also be used for random-access rendering in the video.
Choose a start KF by left-click onto it –you will see an left-arrow will be put under it and an end KF
again by left-click onto another –a right-arrow is placed under it. Then by pressing the display
button in the middle of the UI screen, you can playback the partial video that is in between the two
arrows. Once the playback is initiated, you can use buttons to pause and stop video clip.
Q: What parameters exist to extract SFs?
A: None except the mode of SF extraction. You can do automatic extraction with no parameters, or
you can give the number of SFs or a threshold value.
Q: Where will the SFs displayed?
A: Right under the KFs. There are 2 SF extraction methods and so you will see 2 lines of SFs
displayed.
Q: But only 7 frames per line are present in the UI, Why?
A: You can see more SFs by pressing “Next” arrow button on the right button corner of the KFs &
SFs window.
Q: How can I make full-screen display of video clips?
A: There are 2 ways of doing this: You can double-click the video window OR you can choose
<Display Functions> Æ <Full Screen Display..> menu and then start playback.
Q: How can I see database images in their original sizes?
A: You can use right click menu on an image and choose <Normal Display> to see the image on its
original size.