Download E.2.2.3.1 Anexo 2 Graphic Annotation Tool (GAT) User`s Guide

ENIT 2007-1012
Tecnologías para la Creación y Gestión Automatizada de Contenidos
Audiovisuales Inteligentes
E.2.2.3.1 Anexo 2
Graphic Annotation Tool (GAT)
User's Guide
Title:
Graphic Annotation Tool (GAT) User's Guide
Author:
Neus Camps and Khristina López
Version
4
Abstract:
This document presents the guide needed to understand and use all
the features of the annotation tool based on regions developed by the
GPI of the UPC.
Keywords
Contents
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
1
ENIT 2007-1012
1 INTRODUCTION.......................................................................................................................4
2 INSTALL, RUN AND UPDATEGAT..........................................................................................6
2.1 JAVA W EB START SOFTWARE.......................................................................................................................6
2.2 GETTING THE JAVA WEB START SOFTWARE.....................................................................................................6
2.3 RUN GAT WITH THE JAVA WEB START SOFTWARE...........................................................................................6
2.4 UNINSTALL GAT.......................................................................................................................................7
2.5 RUN GAT WITH THE JAVA VIRTUAL MACHINE.................................................................................................7
3 USER INTERFACE...................................................................................................................9
3.1 TOOLBAR...............................................................................................................................................10
3.1.1 File menu.................................................................................................................................10
3.1.2 Demo menu.............................................................................................................................10
3.1.3 Help menu...............................................................................................................................10
3.2 ONTOLOGY CREATION PANEL.......................................................................................................................11
3.2.1 Toolbar.....................................................................................................................................11
3.2.2 Edition tree...............................................................................................................................12
3.3 IMAGE TAB.............................................................................................................................................14
3.3.1 Toolbar.....................................................................................................................................14
3.3.2 Image panel.............................................................................................................................17
3.4 COLLECTION TAB.....................................................................................................................................17
3.4.1 Toolbar.....................................................................................................................................18
3.4.2 Thumbnails panel....................................................................................................................18
3.5 ANNOTATION TAB.....................................................................................................................................18
3.6 SEMANTIC PANEL.....................................................................................................................................18
4 MPEG-7/XML ONTOLOGY EDITION.....................................................................................22
4.1 ONTOLOGY CREATION...............................................................................................................................22
4.2 ONTOLOGY EDITION..................................................................................................................................22
4.3 ONTOLOGY WRITING ................................................................................................................................24
4.4 ONTOLOGY READING.................................................................................................................................24
5 ANNOTATION ........................................................................................................................25
5.1 GETTING THE INPUT VISUAL DATA.................................................................................................................25
5.2 ONTOLOGY IMPORT..................................................................................................................................26
5.3 SEQUENCE ANNOTATION............................................................................................................................27
5.4 IMAGE ANNOTATION..................................................................................................................................28
5.4.1 Atomic annotation....................................................................................................................28
5.4.2 Selecting the suppor area........................................................................................................30
5.4.2.1 BPT navigation......................................................................................30
5.4.2.2 Advanced BPT navigation.....................................................................31
5.4.2.3 BPT node identification.........................................................................32
5.4.2.4 Points.....................................................................................................32
5.4.2.5 Lines......................................................................................................33
5.4.2.6 Rectangles contour................................................................................33
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
2
ENIT 2007-1012
5.4.2.7 Rectangles area......................................................................................34
5.4.3 Composite annotation..............................................................................................................36
6 SEMANTIC NAVIGATION .....................................................................................................37
6.1 VISUALIZING INSTANCES.............................................................................................................................37
6.2 REMOVING INSTANCES...............................................................................................................................39
6.3 IMAGE FORMAT INSTANCES.........................................................................................................................39
6.4 WRITING
INSTANCES.................................................................................................................................39
6.5 READING INSTANCES.................................................................................................................................39
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
3
ENIT 2007-1012
1 Introduction
This document presents the user's guide, necessary to understand and use all the
functionalities of the Graphic Annotation Tool (GAT) developed by the Image Processing Group
of the UPC in the Activity 2 Task 3 of the i3media Project.
The goal of GAT is to provide a graphic annotation interface of key-frames in order to make
easier the following automatic creation of semantic models1. It has been designed and
implemented to provide the user with an intuitive and interactive interface through using the
keyboard and mouse.
The application requires three different types of input data related to the visual content:
–The picture in any standard format (JPG, PNG ...).
–The thumbnail image in any standard format (JPG, PNG ...).
–An initial partition of the image previously generated through a process of segmentation.
–The partition tree (PT) by defining an additional set of regions formed as a combination of the
regions of the initial partition.
The access to the three types of visual data sources is done through a single MPEG-7/XML file
called BPT (Binary Partition Tree), which contains the parent-child relationship between nodes
PT as well as references to image files, the thumbnail and the original partition. The user can
also use the file called VD (Visual Descriptors, visual descriptors), which contains the visual
descriptors and references to image files, the thumbnail, the original partition and the BPT.
The semantic objects to annotate, and their component parts, are identified and structured in
one or more ontologies of semantic classes. The objects and their parts are connected by the
relation "part of". The relationship "part of" between semantic classes is indicated in the marking
process. Each semantic class is characterized by a text label and a numeric identifier. The
current version supports reading of ontologies defined in MPEG-7/XML or TXT file, or imported
from a URL, i.e. a web page. The application allows import ontologies in OWL (Web Ontology
Language) format. The interface also includes an ontology editor MPEG-7/XML format.
The tool allows the semantics contained in an image may be annotated on two scales: image or
pixel. In the case image, the support area is the entire image, while in the case pixel, the area of
support is a subset of image pixels, a region. The support area selection by the user depends
on the nature of the semantic class. While some abstract concepts such as forest or sport event
will want to be represented with all the pixels of the image, other concepts such as car or
football player can be represented with a specific subset of image pixels.
A pixel scale techniques can be used for the selection of support area are divided into two
groups. One option is based on drawing points, lines and rectangles, giving approximate
descriptions of the objects without determining the exact pixels that represent them. The second
option is based on the selection of a set of regions, connected or not, of the partition generated
from the image, defining the pixels representing the objects. This selection is performed by
navigating the PT. The selection technique used by the user depends of the accuracy required
in the final application.
The result of the annotation of images, is the creation of instances of a semantic class. The
semantic point of view distinguishes between two types of instances: atomic or composite. On
the one hand, the area support of an atomic instance represents a single semantic class. On the
other hand, the support area of a composite instance not only represents a semantic class but
also describes its parts. A composite instance is only possible if its parts have been annotated.
There is no limitation in the semantic decomposition levels supported. Thus an instance of a
semantic class that is part of a top-level semantic class can also be decomposed into instances
1 A semantic class is a set, group or collection of objects. Its name describes a semantic meaning for the
objects.
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
4
ENIT 2007-1012
of other subclasses. For example, the semantic class head, could be a part of the semantic
class person and at the same time, it could be decompose into semantic classes face and hair.
On pixel level are possible atomic and composite instances, but on image level, only atomic,
since the relation "part of" can not be established between images.
The GAT allows massive image annotation on image level, i.e., instead of frame by frame
annotation, the user can annotate series of images at the same time, resulting an atomic
instance for each of the images.
The GAT output file corresponds to an annotation session and is also written in MPEG-7/XML
format. The file describes, for each annotated image, which support area (either the whole
image, regions, points, etc..) represents the instances of semantic classes defined in an
ontology.
The application also includes built-in user manual in HTML format and demonstrations of its
operation.
GAT is an application implemented in Java, and its execution requires the previous installation
of Java over the operating system, which could be MacOS, Microsoft Windows o GNU/Linux.
The images that appear in this document are property of TVC, Televisió de Catalunya, SA,
these images contain copyright and are provided by the TVC with the exclusive purpose of
investigation for the i3media Project.
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
5
ENIT 2007-1012
2 Install, run and update GAT
GAT can be executed using the Java Web Start technology of Java.
2.1 Java Web Start software
The Java Web Start software permits download and run Java applications from the Web:
●This software permits to activate the applications with a click.
●This software guarantees the last version of the application.
●This software avoids a complex installation and update process.
2.2 Getting the Java Web Start software
The Java Web Start software is needed to run GAT. If this software is not installed, the
application redirects your browser to the Java Web Start installation page. The Java Web Start
software is a component of the Java Runtime Environment (JRE). To check if the last version of
Java is installed, look for the next link:
http://java.com/es/download/installed.jsp
From this site, it will be recommended the link to download and install the last version of Java. It
also can be done from the next link:
http://java.com/es/
The first time GAT is executed from the Java Web Start technology, the software will be
downloaded and saved locally, in the cache memory of the computer. In this way, the following
executions will be instantaneous, because the necessary resources are kept.
Every time the application is initiated, the Java Web Start software check in the Web for the last
available version of the application; if a new version exists, this will be downloaded and run
automatically.
2.3 Run GAT with the Java Web Start software
The Java Web Start technology offers several methods to run the application on any operating
system. This methods are described below.
From a Web browser
Click on the link indicated in the next Web:
http://gps-tsc.upc.es/imatge/i3media/gat/index.html
From an icon on the Desktop or the Start menu (only Microsoft Windows)
If the application will be used frequently, you can create a shortcut in the Desktop or in the Start
menu. Java Web Start asks for the creation of shortcuts or an entry in the Start menu. If the
answer is “yes” all the following executions of the application may be initiated without a browser.
From the Java applications cache browser (only Microsoft Windows)
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
6
ENIT 2007-1012
Java Web Start also provides an applications cache browser, it can be ran from the Control
Panel of Java. This browser permits to run the downloaded applications directly.
Instructions to run the application cache browser in Microsoft Windows:
•Go to Start > Control Panel. Double click on Java icon. The Java Control Panel will start.
•Click on the General tab.
•Click the View button in the Temporary Internet files.
•Click on GAT in the applications list.
From the command window of Windows or the console of Linux
GAT can also be run from command window of Windows or from the console of Linux writing:
javaws http://gps-tsc.upc.es/imatge/i3media/gat/javawebstart/gat_ca.jnlp
2.4 Uninstall GAT
To uninstall GAT in Microsoft Windows, run the cache browser, choose with a right-click the
application and select the Remove option in the appearing menu.
2.5 Run GAT with the Java Virtual Machine (JVM)
To run the application with the Java Virtual Machine, go into the project folder and run the
following command:
Microsoft Windows
java -cp ./bin/;./lib/images.jar;./lib/help.jar;./lib/jsr173_1.0_api.jar;./lib/MPEG7Ver1.jar;./lib/substance.jar;./lib/xbean.jar;./lib/jhall.jar;./lib/jsearch.jar;./lib/jh.jar;./li
b/jhbasic.jar edu.upc.tsc.gps.gpi.gat.Gat
Linux
java -cp ./bin/:./lib/images.jar:./lib/help.jar:./lib/jsr173_1.0_api.jar:./lib/MPEG7Ver1.jar:./lib/substance.jar:./lib/xbean.jar:./lib/jhall.jar:./lib/jsearch.jar:./lib/jh.jar:./li
b/jhbasic.jar edu.upc.tsc.gps.gpi.gat.Gat
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
7
ENIT 2007-1012
3 User interface
The Figures 1 and 2 show two different screenshots of the user interface GAT, containing a
toolbar at the top of the window, the tabs Annotation, Image and Collection occupying the
central area and the left side of the window, and the Semantic Panel located on the right side of
the window.
Figure 1: User interface
Figure 2: User interface
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
8
ENIT 2007-1012
3.1 Toolbar
he toolbar contains three menus, File, Demos and Help, each one with several options.
Inactive options appear in grey.
3.1.1 File Menu
The File menu contains options for reading and writing ontologies, and for the configuration of
the application preferences.
New (new annotation)
Allows create or import an ontology from a local file or Web address (URL). The option to cretae
a new ontology opens the Ontology Editor Panel.
New
Open (annotation)
This option allows to open an ontology file saved previously.
Open
Save
Allow to save the current annotation.
Save
Preferences
This option opens a configuration panel. This Panel allows to change the application language
and the user's directory to save the results which appears by default.
It also allows to introduce the authentication data to connect to the remote server for extracting
files.
3.1.2 Demo Menu
The Demo Menu contains options with demonstrations of the application.
3.1.3 Help Menu
The content of the Help menu is the following: GAT's Help and About GAT.
GAT's Help
This option starts the HTML Help system of GAT.
GAT's Help
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
9
ENIT 2007-1012
About GAT
This option shows a dialog with information about GAT.
3.2 Ontology creation panel
The MPEG-7/XML Ontology Creation Panel allows the creation of new ontologies. Access to
this panel through the option New in the File Menu, and choosing Create new ontology.
The panel contains a toolbar and a edition tree. Output files are MPEG-7/XML. Figure 3 shows
a screenshot of the panel.
Figure 3: Ontology creation panel
3.2.1 Toolbar
The toolbar includes the buttons for the creation of MPEG-7/XML ontologies.
New Class
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
10
ENIT 2007-1012
This option is enable only when the selected tree node is the root associated to the ontology.
Allows to insert a class to the ontology, adds a descending node and enables the label text
edition via the keyboard.
New Class
Rename
This option is enable only when the selected tree node is a class node. Allows to rename a
class through its text label.
Rename
Remove
This option is enable when the selected tree node is a semantic class. Allows to remove the
class, by removing the node.
Remove
3.2.2 Edition tree
The edition tree shows vertically the current ontology. The root node of the tree represents the
entire ontology. The child nodes represent the classes and their text label is the name of each
class.
The nodes can be selected individually. To select a node, left-click above it.
At last, the panel allows to select a directory to save the file with the created ontology and
concede it a name.
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
11
ENIT 2007-1012
3.3 Image tab
The image tab allows to annotate images at pixel and image scale. Contains a toolbar in the
upper part and an image panel above. The Figure 4 Shows a screenshot of the tab.
Figure 4: Image tab
3.3.1 Toolbar
The toolbar includes the components to select the support area to annotate. The disabled
buttons appear in grey.
Open
Allows to open a BPT or VD file, or a standar image (JPEG,PNG,etc.).
Open
Path (drop down list)
Drop down list to select opened file. The first option of the list is the last file read.
Path
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
12
ENIT 2007-1012
Zoom In, Zoom Out and Original Size buttons
Increases, decreases and returns to the original value the display size of the image,
respectively.
Zoom In
Slider Mask Transparency
Changes the transparency of the covering mask between de 20% and the 80%.
Slider Mask Transparency
Mask Color
Drop down list to select the color of the covering mask. Nine different colors are available.
Mask Color
Delete
This tool deletes the selected support area.
Delete
BPT node identifier annotation
This option activates the normal region annotation showing the BPT node identifier.
BPT node identifier
BPT navigation annotation
This option activates the normal region annotation through the BPT navigation.
BPT navigation
BPT advanced navigation
This option activates the advanced region annotation through BPT navigation.
BPT advanced navigation
Rectangle to BPT
Activates the technique to select regions by drawing a rectangle to visualize all the regions
contained within. Allows to select or deselect new regions marking a point.
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
13
ENIT 2007-1012
Scribble to BPT
Enables the technique to markup regions as support area by drawing a scribble marking the
background or the object.
Markup background
Markup object
Annotation using rectangles
This option activates the annotation based on rectangular area.
Rectangle area
Annotation using lines
This option activates the annotation based on lines.
Line
Annotation using points
This option activates the annotation based on points.
Point
3.3.2 Image Panel
The image panel is where the image to annotate is displayed. It is the workspace that allows to
annotate at image and pixel scale.
3.4 Collection tab
The Collection tab is the workspace that allows to annotate a strip of images at once and view
the support area that represent the instances associated with these images. It contains a toolbar
at the top and a panel below. Figure 5 shows a screenshot of the tab.
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
14
ENIT 2007-1012
Figure 5: Collection tab
3.4.1 Toolbar
Open
Open a file or directory of BPT, VD or image files in standard format (JPEG, PNG, etc.).
Open
Path
Drop-down list to select a file or directory opened previously. The first option on the list
corresponds to the last file or directory read.
Path
Markup Positive
Mark images as positive annotation one by one.
Markup Positive
Mark Negative
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
15
ENIT 2007-1012
Markup images as negative annotation one by one.
Markup Negative
Markup Neutral
Markup images as neutral annotation one by one.
Markup Neutral
Markup All Positive
Markup all the sequence as positive annotation.
Markup All Positive
Markup All Negative
Markup all the sequence as negative annotation.
Markup All Negative
Markup All Neutral
Markup all the sequence as neutral anotation.
Markup All Neutral
3.4.2 Thumbnails Panel
The thumbnails panel displays a scaled version of the images to annotate. This workspace is
used to develop atomic annotations at image scale.
3.5 Annotation tab
The annotation tab is the workspace that allows users to review the instances of the ontology
that is being modified and delete the wrong instances. All images appear annotated with its
corresponding annotation, whether at the image or pixel level, marked with a color mask. This
tab is closely linked with the panel as explained in the next section.
3.6 Semantic panel
The Semantic Panel allows the exploration of semantic class instances. It contains a toolbar
and a field of autocomplete words in the top and a panel that contains the semantic tree, lower.
By right-clicking the panel, a pop-up menu appears. Figure 6 shows a screenshot of the tab.
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
16
ENIT 2007-1012
Figure 6: Semantic panel
3.6.1 Semantic tree
The semantic tree shows the results of the current annotation session. Contains many subtrees
as ontologies imported. The root of a tree represents all the classes contained in an ontology
and its label text, is the name of the ontology. The second-level nodes represent semantic
classes and their label is the name of the class. The third level nodes represent instances,
atomic or composite. From the fourth level, the nodes represent parts.
3.6.2 Autocomplete field
The application includes a word autocomplete field that allows users to search over ontologies
and tree classes. Each time a letter is typed, automatically, the word is completed from the
available words that match the typed letters. If more than one word exists, that fits the written
letters, the words available are listed alphabetically in a drop down list. If there is no word, it
beeps.
To accept the suggestion, press Enter when appears the suggestion, or select it from the list.
The corresponding ontology or class will be selected in the tree. If you disagree with any
suggestion you can continue writing without problems. The suggestions are presented as
shaded text. Figure 7 shows an example of autocomplete.
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
17
ENIT 2007-1012
Figure 7: Autocomplete field
3.6.3 Drop down menu
New Instance
The option is enabled only with the Image tab and when the selected node represents a
semantic class. Creates a new empty instance of the class, adding a descending node.
Copy
This option is enabled only with the Image tab and when the selected node represents an
atomic instance. Copies the instance to the clipboard of the application. The clipboard can only
store one instance.
Keyboard: Ctrl+C
Paste
This option only works with the Image tab and when the selected node in the tree represents an
empty instance (New Instance text label) or an instance or a composite instance. Copies the
clipboard content as a part of the selected instance, adding a descending node.
Keyboard: Ctrl+V
Remove
Option enabled when the selected node represents an instance. Allows to remove the instance,
removing the node.
Keyboard: Del
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
18
ENIT 2007-1012
Save Instance as Image
Option enabled when the selected node represents an instance. Allows to save the support area
representing the instance as a standard image format.
Save Mask as Image
Option enabled when the selected node represents an instance. Allows to save the support area
represented by the mask, as a binary image: white color for the selected area, black for the rest.
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
19
ENIT 2007-1012
4 MPEG-7/XML ontology management
MPEG-7/XML ontology creation is performed in the Ontology Editor Panel and its
management is performed in the Semantic Panel.
4.1 Ontology creation
To create a new MPEG-7/XML ontology, choose the option New ontology from the pop-up
menu that appears by selecting New from the File menu or the toolbar. In case of a ontology
opened previously, a dialog asking whether to save the changes appears. It will create a new
tree only with the root issue, triggering the edition of their text label. Enter the desired name and
press the Enter key. After saved, the ontology appears in the Semantic Panel.
4.2 Ontology edition
The edition is to add, rename or delete semantic classes. To add a class, select the ontology
and choose Insert from its menu. A new node will be added. Enter the desired name and press
the Enter key.
The names of the classes can not be empty, can not be repeated and may not be equal to the
name used by the ontology. If you enter an invalid name, you get an error dialog (Figures 8, 9
and 10).
Figure 8: Entering an empty name
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
20
ENIT 2007-1012
Figure 9: Entering a name used by other semantic class
Figure 10: Entering a name used by the ontology
To change the name of a class, select the class and type the new name in the text field located
on the top of the edit panel. If you enter an invalid name, you get an error dialog as above.
To remove a semantic class, select it and choose the Delete option from the drop down menu
or press the Delete key. You will get a confirmation dialog to accept the changes (Figure 11).
Figure 11: Confirmation dialog to remove a semantic class
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
21
ENIT 2007-1012
4.3 Ontology writing
Click the Save button on the toolbar to save the ontology in a MPEG-7/XML file . If you have not
saved the ontology Save dialog will appear, allowing to enter a name for the file and select the
location on the disk. Click the Save As button to change the name and location.
4.4 Ontology reading
Click the Open button on the toolbar to open an ontology from disk. The Open dialog appears,
allowing to select a MPEG-7/XML ontology file. If the selected file is not valid, appears an error
dialog (Figure 12).
Figure 12: Not valid ontology file
If the selected file is valid, the ontology is loaded into the tree. If another ontology is open,
appears a dialog asking whether to save the changes.
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
22
ENIT 2007-1012
5 Annotation
At first, the most practical, is to display the image or images (in case of sequences) to annotate
and then import the ontologies that describe the semantic concepts present in the image (or
images). As explained in the introduction, some concrete visual data is needed.
5.1 Getting the input visual data
One option is to obtain the input visual data from a BPT or a VD.
The reading of BPTs and VDs is performed from the Open button from the toolbar and the File
menu.
For sequence annotation, can be opened a file or directory. If the selected files are valid, the
thumbnails are displayed in the thumbnail panel without the possibility of resizing.
In the case of image annotation, only one file can be opened. If this is valid, the image is
displayed at its original size in the image panel, but in this case, can be resized through the
zoom buttons on the toolbar.
If a selected file is not valid, you get an error dialog as in Figures 13 i 14.
Figure 13: Not valid BPT file
Figure 14: Not valid VD file
A second option, if BPT or VD file is not available, is to open images in standard format and
upload them to a remote server. This server will execute a series of applications that generate a
thumbnail, a partition and a BPT. The four files were stored on the server and the application
may recover them. This process will end up getting input visual data necessary to make the
annotations. This operation is completely transparent to the user.
Is possible to open more than one file and directory during the session. When you open a BPT,
a VD or image, the program extracts the necessary data (image, thumbnail, partition, PT) and
stores them in memory. In this way, if you open a new file or directory, the program does not
need to extract the data again because the have been stored in memory. It is an important
resource savings. If that is the case, an informative dialogue appears.
Opened files and directories during the annotation session are available on the drop-down list
on the toolbar tabs Image and Collection. The first element corresponds to the last file or
directory opened. If you expand the list and select an item, the image or thumbnails will be
displayed.
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
23
ENIT 2007-1012
5.2 Ontologies import
To import an ontology, select New option from the File menu or the general toolbar, and choose
the Import Ontology option from the pop-up menu. Appears a dialog Open, allowing to select an
MPEG-7/XML or TXT ontology file format. If the selected file is invalid, an informational dialog
will appear (Figure 15).
Figure 15: Error dialog when the selected ontology was already
imported
You can import as many ontologies as needed. Figure 16 shows a screenshot of the semantic
tree after a few ontologies have been imported.
Figure 16: Semantic tree with the ontologies
imported during an annotation session
You can delete an imported ontology with the Delete option from the drop down menu or by
pressing CTRL + V. If any class is instantiated, the instance is deleted from the annotation
session. Before, appears a dialog to confirm the operation.
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
24
ENIT 2007-1012
Figure 17: Confirmation dialog to delete the selected ontology
5.3 Sequence annotation
The cycle of sequence annotation of images is simple.
To annotate an image sequence follow the next steps:
1)Select the semantic class in the right panel.
2)Select the annotation type: positive, negative or neutral by selecting the icons (green, red and
yellow) in the toolbar of the tab.
3)Select the image with one click above. This will draw a border color of the type of instance.
4)Click the right mouse button on the panel to validate the selection or click again on it to
deselect it.
To select more than one image, just click on the pictures to annotate them.
To select all the images in the strip, choose the option to mark all images as positive, negative
or neutral in the toolbar.
The Figures 18 and 19 show screenshots of the tumbnails panel during the annotation.
Figure 18: Images selected
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
25
ENIT 2007-1012
Figure 19: Images annotated
Once the selection is validated, the instances will be added to the semantic tree.
5.4 Image annotation
The cycle of atomic and composite annotation is different.
5.4.1 Atomic annotation
To make an atomic annotation follow these steps:
1) Select in the semantic tree the class you to instantiate.
2) Right-click the button on the tree and choose New instance from the menu. This
adds a new instance of the selected class. This instance is empty and has as New
Instance text label (Figure 20).
Figure 20: New empty instance in the semantic tree
3)Go to the toolbar and choose a selection technique.
4)Place the cursor over the image and start the selection of the support area under
arrangements of the used technique. The selection techniques are described in detail in
Section 5.4.2.
When you start the selection, the image will be covered with a mask with selectable
color and transparency in the toolbar (Figure 21). Select the color and transparency as
the most appropriate for the image.
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
26
ENIT 2007-1012
Figure 21: Starting the selection
Continue the selection with the support area until it meets the desired instance (Figure
22).
Figure 22: The support area meets
the desired instance
The toolbar includes the Clear button to erase the current selection and start again. If the
technique is changed after start the selection, the effect is the same. The mask
disappears when you start again.
5)Confirm the selection with a right-click on the image. If no selected area, the instance
represents the entire image (image level annotation), on the other hand, the instance will
represent the support area selected (pixel level annotation). The new instance is added to
the semantic tree.
5.4.2 Selecting the support area
5.4.2.1 BPT Navigation
Select the BPT navigation tool to activate this annotation mode. The selection process begins
with a click on a pixel of the image. After that, the area around this pixel is selected, covering
these pixels with the transparency mask and showing the pixels from the original image. If the
selected region does not correspond to the instance to annotate, the connected regions can be
selected by moving downwards the mouse wheel.
Figures 23 and 24 show the same example with an initial region selected and expanded after
the downward movement of the mouse wheel until get the specific set of selected regions. The
expansion of regions can grow to fill the entire image.
Figure 24: Set of connected regions
selected
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
Figure 23: Initial region selected
27
ENIT 2007-1012
If the selected regions exceed the expected instance, the upward movement of the mouse
wheel allows deselection of regions, pixels of which are covered with the mask. The deselection
can reach the initially selected anchor point, again showing all covered with the mask image,
with no region selected.
If the instance is made up of areas not connected, you can select and expand a new anchor
point by clicking the left mouse button on a non-transparent pixel of the image (Figure 25).
Figure 25: Two sets of regions not
connected
Is possible to switch selected regions to not to selected by left-clicking on one of its transparent
pixels. After clicking the pixels,the region will be covered with the mask.
5.4.2.2 Advanced BPT navigation
To activate this annotation mode select the Advanced BPT Navigation tool. This method is an
improvement of the BPT navigation method (see section 5.1.1). This option makes easier the
selection of sets of connected regions with similar number of pixels.
Moving the mouse cursor over the image, the region around the cursor position cover its pixels
with a transparency mask, showing the pixels of the original image. Moving downwards and
upwards the mouse wheel, the connected regions are shown or hidden, but in this case, these
regions are not selected yet. Increase the regions expansion since have the desired area
(Figure 26). Once the search area is defined, move the cursor out of the initial region. The
previous regions will hide and a new set of connected regions will be shown around the new
position. In this way, moving the cursor on the image, the sets of connected regions will be
shown with a similar area (Figure 27).
Figura 26: Set of connected regions
with a search area
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
Figura 27: Set of connected regions
with a similar area
28
ENIT 2007-1012
When the connected regions are adjusted to the instance or part of the instance, select them by
clicking on the image. Once the selection is done, the search area will be maintained, and other
sets of connected regions with a similar size can be selected.
To define a new search area, move the mouse wheel to hide the regions since obtain only one
region, associated to the cursor position. A new regions expansion can be initiated in the same
place or from other position.
When the instance is adjusted, validate the selection with a right-click.
5.4.2.3 BPT node identification
To activate this mode, select the BPT node identification tool. To enter the node identifier,
write the text in the box beside and press Enter. The regions associated to this node will be
selected. If this identifier was entered before, the regions will continue selected. If the identifier
is empty, an exclamation sound will indicate an error. To validate the selection, right-click on the
image.
This annotation mode works independently from the others. When this option is activates, the
annotation process is initialized, removing the past selections. The annotation process is also
initialized when another annotation mode is activated.
5.4.2.4 Points
To activate this option, select the Point tool. To select a point, click on the image. This point will
showed like a red cross on the mask.
To edit one point, choose the Select tool and click over the defined area, the cross. To delete it,
press the Del key, and to move it, drag this point to the new position.
The Figure 28 illustrates a example of a set of points.
Figure 28: Annotation using points on the
goal
5.4.2.5 Lines
To activate this annotation mode, select the Line tool. To draw the line, click on the image and
move the cursor, to end the line, double-click. The lines appear opaques with the color of the
mask.
To edit a line, choose the Select tool and click on the line. To delete the line, press the Del key.
To move the line, drag it to the new position. To change its size, drag one of the vertex.
The Figure 29 shows an example of annotation using several lines.
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
Figura 29: Annotation using lines in the
goal area
29
ENIT 2007-1012
5.4.2.6 Rectangles
To activate this annotation mode, select the Rectangle tool. To draw the rectangle, drag the
cursor and after, release the button. The rectangle area appear like a transparent area in the
mask and its contour appears opaque with the color of the mask.
To edit a rectangle, choose the Selection tool and click on the rectangle. To delete it, press the
Del key. To move the rectangle, drag it to the new position. To change its size, drag one of the
borders or vertex.
The Figure 30 shows an example of annotation using several rectangles.
Figure 30: Rectangles area selection
5.4.3 Composite annotation
To make an composite annotation, first annotate the parts. Then follow these steps:
1)In the semantic tree, select the class to instanciate.
2)Right-click over the tree and choose New instance from the menu. This action adds an
instance of the class selected. This instance is empty and has text label New Instance.
3)In the tree select an atomic instance for the part.
4)Right-click the tree and choose Copy from the drop-down menu or use the key combination
CTRL + C.
5)In the semantic tree select New Instance.
6)Right-click the tree and choose Paste from the drop-down menu or use the key combination
CTRL+V. The part is added to the New Instance and the text label will be updated, making the
empty instance into a new instance composed.
Repeat the process of copying and pasting atomic instances until the composite instance is
complete.
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
30
ENIT 2007-1012
6 Semantic navigation
If the Image tab is the active, the semantic tree will only show the instances associated with the
image displayed (Figure 29). In the case of Collection tab, the tree will only display the
instances associated to the thumbnails displayed. (Figure 30).
6.1 Visualizing instances
To see the support area that represents an instance, select the instance in the tree. In the
Collection tab, the support area is always the complete picture and this will be covered by a
transparent mask (Figure 31).
Figure 31: Visualizing an instance associated with an image of the sequence
In the Image tab the support area will look different depending on the selection technique used.
The region is covered by a transparent mask (Figure 32), the same as that used during the
annotation cycle. The area of rectangles become covered by the mask as well. The points, lines
and rectangle contours are drawn on the image with the same color as the mask.
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
Figure 32: Visualizing an instance associated with an image
31
ENIT 2007-1012
To view together the support area of all the instances of a class, select the class in the tree
(Figure 33).
Figure 33: Visualizing all the instances of the class
Is possible to change the color and transparency of the mask to improve the visualization.
6.2 Removing instances
To delete an instance, either atomic, composite or a part, select it in the tree and choose Delete
from the drop-down menu or press the Delete key. You will get a dialog to confirm your
decision. (Figure 34).
Figure 34: Confirmation dialog to delete the
instance
6.3 Image format instances
The support area that represents an instance can be saved in standard image format. Select the
instance in the tree and choose Save Instance As Image from the menu. Save dialog will
appear, allowing you to enter a name for the file and select the format (JPEG or PNG) and the
location on the disk. Figure 35 shows an image that corresponds to an instance.
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
32
ENIT 2007-1012
Figure 35: Image that corresponds
to an instance
6.4 Writing instances
Click the Save button on the toolbar to save the results of the annotation session. If this is the
first time, you get the Save dialog, where you can type a name for the file and select your
location on the disk. Click the Save As button to change the file name and location.
6.5 Reading instances
Click the Open button on the toolbar to open a file from a previous recording session. Open
dialogue will appear, allowing to select an annotation file in MPEG-7/XML format. If the selected
file is not valid, you get an error dialog.
If the selected file is valid, will be charged the annotation in the semantic tree and the
thumbnails and images on the tabs Image and Collection respectively.
2010 E2.2.3.1 Graphic Annotation Tool (GAT)
User's Guide
33