Video analytics enhances security picture

Video analytics can tell whether someone is moving too fast in a flow of people, identify an abandoned package or a person loitering. Soon, it will be able to indicate if someone is about to do something they should not.

The trouble with security video is that there’s too much of it and most of it is extremely boring.  Nobody in their right mind wants to look at hours of nothing much happening. So how can one deploy video so that it really does help ensure greater security?

The answer is video analytics, but those in the business are quick to point out that too much was expected too soon.  Courtney Dillon Pedersen, communications manager for Milestone Systems, which makes the XProtect Analytics framework on which different cameras and analytic tools can be integrated over IP, says, “Video analytics was hyped too much at the start.” And Johan Paulsson, CTO of camera-maker Axis Communications, says: “You can’t use video analytics for mission-critical activities.”It would be too risky to rely entirely on a system without human intervention to stop intruders in a high security site, for example.

So what can you use it for? “It helps operators,” says Paulsson. “It adds value to video,” says Pedersen.

Let’s be a bit more precise:  Rasmus Crüger Lund, a system architect with Milestone, says, “Video analytics can assist operators during monitoring by focusing their attention on the correct places. It filters out the stuff that operators do not need to care about, so they can react quickly when something does happen.”

The “hype” to which Pedersen refers was the claim that video analytics could determine which incidents really were a threat to security.  It still can’t do that, but it’s getting closer.  Analytics can tell whether someone is moving in the wrong direction or too fast in a flow of people, identify an abandoned package or a person who seems to be loitering, tell if someone tries to sneak through a turnstile behind someone else, or identify license plates. And of course it can easily tell whether someone has entered an area which no-one is supposed to enter.

But it can’t tell whether these incidents are significant or not. It can only alert an operator who will have to take the decision. These “false alarms” are not really false: an operator looking at the screen would probably have looked more closely at such incidents, only to decide that no action was necessary.  But they are irritating, and too many of them may destroy confidence in the system.

Paulsson believes that one key lies in staff training. “Video analytics are definitely better than human observers at three o’clock in the morning, who are probably not reliable at all,” he argues. “With analytics, you wait for the alarm. Sometimes it will be a false alarm, but you have to manage staff expectations on that.”

James Orwell, head of the Visual Surveillance Research Group at Kingston University in London, thinks one reason for false alarms is the primitive infrastructure of many video installations. “You often have poor resolution, with an analogue feed being digitized and poorly compressed,” he says. “Compared to computer-generated graphics in feature films, we’re still in the Stone Age.”

But things are getting better. Paulsson says high-resolution video is making it easier to design good applications. 

Lund sees complex setting-up as another reason for problems, but he sees a change coming: “In the future, systems will install themselves, or at least tell the operator which parameters they can’t work out on their own,” he forecasts. “The best systems will monitor scenes and learn for themselves what normal activity is.”

As analytics get better, they’ll become more precise in identifying security-relevant situations. Lund observes with interest the research going on at universities on the detection of erratic behavior. “There’s work being done on small facial movements and ways of walking which indicate that someone is about to do something they shouldn’t,” he says.

So far, such detailed detection requires very controlled conditions.  Biometric identification still requires people to stand still when looking at the camera. “Algorithms are still too simple,” says Lund, “but people are working on 3D identification using several cameras.”

At Kingston University, a team is currently trying to identify people who may be carrying guns. Jean-Christoph Nebel told the videoanalytics.info website that people do not regularly carry guns, at least in Europe, so that when someone does so, “it is very usual for your emotional state to be altered.”

Experienced security operators will be asked to look at real video footage of criminals just before they committed gun crimes and footage of ordinary people, and to note whenever they get a feeling that something is suspicious, even if they don’t quite know what it is. The cues will then be combined in a machine learning system.

Orwell believes machine learning has plenty of potential. “If operators can repeatedly label what they see as suspicious, that data can be used to train algorithms,” he says. “And since you don’t have to identify why you think something is suspicious, you can include subconscious elements.”

Increased processing power is what is making this kind of analysis possible. Cameras themselves have so much processing capacity that much of the analysis can be done “at the edge” – within the camera on uncompressed material. And modern compression means that even compressed material remains highly usable. “There’s a general acceptance that analytics will have to be done on compressed material,” says Orwell.

But, especially in post-event analysis, the new challenge is metadata: the data carried with the video signal that tells what it contains, and enables it to be found efficiently. “This is going to be huge,” Lund says. “You’ll be able to find video without re-analyzing it, since the metadata will be in a database. You’ll be able to hunt for all video featuring a certain make of car, or showing a person crossing a particular line.”

Two trade organizations, PSIA and ONVIF, are working on the establishment of metadata standards (see related articles). “The two organizations haven’t agreed,” says Lund, “but two standards are better than 200.” Or than no standards, some might add.

Video analytics is expected to grow fast after its initial hesitant start. IMS Research says the market was worth US$47.5 million in 2008, and it expects average annual revenue growth between 2010 and 2013 of 24 percent. Some of that will be in non-security areas, such as analyzing consumer behavior in stores to see how changes in design influence purchasing decisions, but much of it will continue to be used to ensure that the good guys stay ahead of the bad.

By Michael Lawton