Intuition is something that denies scientific explanation. It’s a feeling and an assumption of what may be coming next based on past experiences and human behavior. A computer vision technique unveiled by the Columbia University School of Engineering and Applied Science in New York City is said to result in a more intuitive sense and higher level of association between people, animals and objects—providing increasingly predictive intelligence and analysis of what will naturally happen next in a sequence of events.
Columbia University’s algorithm
In the study “Learning the Predictability of the Future,” researchers deployed a mathematical framework algorithm to better predict human behavior and “coordinate their actions with ours,” according to Carl Vondrick, assistant professor of computer science at Columbia and the study’s director. Findings from the research were presented this summer to the International Conference on Computer Vision and Pattern Recognition, and its results “open a number of possibilities for human-robot collaboration, autonomous vehicles and assistive technology,” Vondrick said.
According to Columbia researchers, the computer vision technique in the study is the “most accurate method to date for predicting video action events up to several minutes in the future.”
The study analyzed thousands of hours of movies, sporting events and other shows, and the system was able to predict hundreds of activities, including handshakes, fist bumps, high-fives and other actions. Previous research in predictive machine learning centered on a single action, and most models were unable to find the commonalities between possible options and actions.
Abstract reasoning
Columbia computer science doctoral students and the paper’s co-authors Didac Suris and Ruoshi Liu took a different approach to form longer-range predictions.
“Not everything in the future is predictable,” Suris said. “When a person cannot foresee exactly what will happen, they play it safe and predict at a higher level of abstraction. Our algorithm is the first to learn this capability to reason abstractly about future events.”
The A.I. model also recognizes that the future is uncertain and is capable of “hedging the bet: the way a person would.”
Computer vision, machine-learning and analytics have become essential to real-time and forensic activity in the physical security industry. Predicting what people will do next is a critical element of proactive detection and deterrence, providing greater situational awareness for targeted risk assessment and response.
Analytics software use is increasingly commonplace in video surveillance, access control and automation. Analytics from biometrics and face detection software are used in personal identification and to assure health and safety compliance. In video surveillance, technologies include license plate recognition and algorithms to identify gathered crowds, objects left behind or even active-shooter detection. Analytics also transcend security into business and operations, assisting retailers with information on store patrons’ traffic patterns, dwell times and line formation at checkouts so managers can provide additional cashiers for a smoother store experience. In warehousing, logistics and automation, analytics identify distribution anomalies or even unsafe working conditions.
Sizing up a situation
The technique could move computers closer to being able to size up a situation and make a nuanced decision, instead of a preprogrammed action, the researchers reported. It’s a critical step in building trust between humans and computers, Liu said. “Trust comes from the feeling that the robot really understands people. If machines can understand and anticipate our behaviors, computers will be able to seamlessly assist people in daily activity.”
As algorithms keep learning, the benefit will be to analytics, proactive detection and the ability to remove biases, a major complaint to facial recognition and biometric applications. The goal is greater deterrence and more accurate security planning and processes—mitigating potential risk wherever possible.
