The prototype for the semantic region-based image annotation tool – short “srimato” – is under construction right now using:
The source code is under developement – you can fork on github
One thing that is still missing is the analysis of a semantic annotation tool for images. So lets have a look at ImageNotion. Unfortunately I cannot find the demo of the system anymore, i think the system is going to be used commercially.
ImageNotion was created by the FZI (Forschungszentrum Informatik) in Karlsruhe. It is a visual technique for the semantic annotation of images and their segments, with so called imagenotions. The systems stands out of the crowd of semantic annotation tools with its easy-to-use interface and by using nice web-2.0-techniques, for example to use drag&drop for the annotation of image regions. A user can point to a special region of an image by drawing a rectangle on the decided region and combining this selected region with an imagenotion. The system integrates the developement of the ontology in the annotation process. If there´s a missing relation, the user can simply define the new relation.
For example, it is possible to draw a relation like the following: You´ve got an image depicting the Eiffeltower in Paris.
So you start draw a rectangle on the region thats depicting the Eiffeltower, and enter a special label in form of a String. Now you can draw further imagenotions by saying, that you can find the Eiffeltower in Paris. Paris is the capital of France. And France is a member of the European Union. And so on.
So the creation of the ontology is left to the user by using simple and intuitive techniques, starting by the defintion of concrete terms like “Angela Merkel” to abstract terms like “politic”, by creating new notions.
Additionally the system provides techniques for the automatic extraction and annotation of images with detecting faces, emotions, gender, objects and text. This methods enables fast and clever annotation of images. If something is incorrect or missing the user has the possibility to refine the result or correct it.
The future developement will concentrate on the developement of a mashup service, loading pictures from external sources and a web service for getting automatic annotations on any image just by sending an URL.
For the structured representation of the annotation MPEG-7 is being used. The description of persons is made by using semantic web standard FOAF.
The system is quite perfect concerning the annotation and ontologies that can be created. As summery for the developement of my prototype
So after reviewing the discussed vocabulary that can be used to represent image metadata I´ve decided to take the following for the implementation of my prototype:
So after analysing two applications I can make a first summary of the main facts concerning the implementation of a prototype:
Any comments and suggestions are welcome
There are lots of nice and interesting tools to annotate pictures with enriching information, that are not focusing on semantic markup but on interactive pictures. So I´ve decided to take a look at Stipple to get a clue about what might be interesting and important to semantic annotation of images and their regions. In my opinion, this service stands out of the crowd (in comparision to services like ThingLink or LUMINATE™) because of its clean design and lovely usability.
Stipple offers a clean and easy to use editor for the annotation of images and their regions. Just by clicking on a part of the image, a editor pops up and all you have to do is inserting any kind of URL. If this URL is related to e.g. wikipedia, vimeo, flickr or youtube, some information is automatically extracted and displayed in the region that has been annotated.
So its really simple to add content and links to your images and whats depicted inside. Additionally Stipple has the concept of supporting affiliates by adding products and shops that directly link to the store, where you can buy for example the shoe that is covered in the picture.
So, Stipple covers all aspects that might be interesting in a picture and a way to spot to related resources in the web.
To cover these information in a semantic meaning would be a great way to enable machines to know whats inside this picture encouraging new possibilities in search and retrieval of those images.
There are plenty of software-tools and projects related to image annotations, already. So I have to analyse these existing solutions to extract nice features or missing features so my prototype application for region-based image annotation would be a great summery of all of them.
At first I will present PhotoRDF, a project published by the W3C in 2002.
It was a project for the demonstration of currently technologies under developement, like RDF Schema or the Jigsaw-Server. The goal was to emphasize the relation and the potential of RDF Schema in cases of metadata and the web.
The system provides three main parts
To sum it up, I will only present the results of the analysis:
Features to be extended:
To come clear about what kind of vocabulay and technologies I will need for the developement of my region-based image annotation prototype I´ve created a wireframe catching the main features, the prototyp should be able to do.
I will explain the main features of the prototype as well as suggestions for implementation.
Main principles of the prototype should be as follows:
So the main question that comes up is how to integrate other vocabulary and ontologies into the Ontology for Media Resources to have a clear base for interoperable metadata?
A brief summery and analysis of each application will follow.
Other thoughts concerning this prototype are future directions like:
Today, this blog has been launched to document the process of my bachelor thesis and to have a communication tool for everybody supporting and following the developement of my thesis.
Please stay tuned – the blog will be filled with further content in the next days.