The World Wide Web is a visual place, full of images and video. As of September 2010, Flickr hosts over 5 billion photos, while over 2 billion videos are watched every day on Youtube. This is a vast store of information, but how can we find what we want?

Currently, images are almost always identified by their associated text, be it a caption or a tag. This has a number of drawbacks: the text must be added manually, and may not describe the image accurately or completely. The vast majority of images are not labelled at all, and are therefore invisible to search engines. Outside the realm of the World Wide Web, there are vast amounts of unlabelled images including personal photo collections, surveillance video, and satellite and aerial imagery.

To fully exploit the potential of images, we need the ability to search for them based on the contents of the images themselves. If I want some pictures of a 1965 Shelby Cobra, I should be able to find the highest quality results regardless of whether they are tagged or not. Going further, if I have a holiday snap of a famous building, I should be able to use the image itself find other pictures of that building, even if I don’t know what it’s called:

This so-called content based image search has been an active area of research for several years now. In the ACVT, we are developing a cutting edge image search implementation, designed to go beyond the world wide web. The project is funded by the Defence Science and Technology Organisation (DSTO), in response to a global problem in defence: far more image and video is captured by satellites, aircraft and troops, than can ever be manually watched, even by an “army” of analysts. We aim to give analysts the tools they need to rapidly search massive amounts of imagery, to answer questions like:

• Where and when have I seen this before?
• I’ve collected weeks of video – what are the most “interesting” clips?
• How has this area changed since I last saw it?

Of course, the same technology can be applied to web image databases to deliver true image based web search. As the project progresses, we plan to launch a demonstration web site so everyone can try it out for themselves!

Using Content Based Image Retrieval to find similar objects in a database.