Making computers see

feat-img-computer-see

Imagery can be of immense value in the defence realm. Whether obtained through surveillance or other means, it can provide critical intelligence about physical environments, structures, objects and equipment, populations and more.

Historically, however, extracting accurate, meaningful information from that imagery has been anything but easy. Although facial-recognition technology has come a long way in recent years, teams of analysts have still had to conduct the vast majority of image analysis by sight, and personally document their findings.

But according to Professor Anton van den Hengel of the University of Adelaide’s Australian Centre for Visual Technologies (ACVT), that’s about to change. His team has developed a system that not only allows computers to interpret images, but clearly and usefully respond to spontaneous questions about them.

“We’ve essentially created an artificial intelligence VQA (visual question answering) system that gives computers the gift of visual communication,” says Anton.

“You can ask our system all kinds of ad-hoc questions about images—like what’s in them, numbers of specific items present, their colours, etc.—in plain English, and it will answer them virtually immediately, also in English.”

The ACVT team, working within the University’s School of Computer Science, is leading the world on all international benchmarks in the VQA field, and are continuing to push their technology’s capabilities further.

“We recently developed a method of bringing in linked, multi-tiered information from an external source so that highly complex questions about images could be answered.

“You could ask, for example, ‘How many mammals are in the image?’, which is difficult because there’s no such thing as a mammal detector. You first have to detect an animal, recognise it, and then look it up to figure out whether it’s a mammal.

“Our system can now do that instantaneously.”

Beyond the obvious advantages for defence applications, the system also has the potential to enhance life throughout our community, says Anton.

“Imagine being able to ask your computer to show you a photo you took on a specific holiday in a specific location, of particular people eating a specific meal. You could find exactly what you wanted in an instant.

“Making computers see will change the way we live.”

Related links:
Prof. Anton van den Hengel
Computer Science research 
Australian Centre for Visual Technologies 
Defence and Security website