Loris Bazzani's Website

Learning attentional policies for object tracking and recognition in video with deep networks, ICML, 2011.

We propose a novel attentional model for simultaneous object tracking and recognition that is driven by gaze data. Motivated by theories of the human perceptual system, the model consists of two interacting pathways: ventral and dorsal. The ventral pathway models object appearance and classification using deep (factored)-restricted Boltzmann machines. At each point in time, the observations consist of retinal images, with decaying resolution toward the periphery of the gaze. The dorsal pathway models the location, orientation, scale and speed of the attended object. The posterior distribution of these states is estimated with particle filtering. Deeper in the dorsal pathway, we encounter an attentional mechanism that learns to control gazes so as to minimize tracking uncertainty. The approach is modular (with each module easily replaceable with more sophisticated algorithms), straightforward to implement, practically efficient, and works well in simple video sequences.

Download Code (v0.4, 30/05/2011)

Download synthetic dataset

• For the instructions, have a look at the README.txt file
• We provide the demo code for the synthetic video sequences.

Some results:

Bibtex:

@inproceedings{Bazzani:2011ICML,
author = {Bazzani, Loris and de Freitas, Nando and Larochelle, Hugo, and Murino, Vittorio and Ting, Jo-Anne},
title = {Learning attentional policies for tracking and recognition in video with deep networks},
booktitle = {International Conference on Machine Learning},
year = {2011},
address = {Seattle, WA, United States},
}