ICONS: Incident Recognition for Surveillance and Security
ICONS is a DTI/EPSRC LINK project that links
Queen Mary Vision Lab with several industry partners including Safehouse technologies, BAA plc, BT and Heritage Protection Ltd.
activities captured in video footages are characterised by the spatio-temporal
structure of their motion pattern. These structures are intrinsically
probabilistic and often ambiguous. In general, they can be treated as temporal
trajectories in a high-dimensional feature space representing correlated visual
measurements. For example, the spatio-temporal structure of a simple behaviour
such as walking towards a telephone box in a public place can be represented by
the trajectory of an observation vector given by the mean position and
displacement of the human body. The
problem becomes hard when human (and other motion object) activities captured in
crowded public space need to be interpreted correctly. The aim of the project is to advance
- Zero motion detection: detection of medium- to long-term visual changes in a scene (eg: deployment of a parcel bomb, theft of a precious item)
- Behaviour recognition: characterise and detect undesirable behaviour in video data (eg: eg: trespassing, patterns of theft, violence)
Zero Motion Detection
The task is to distinguish background from foreground, even though the scene appearance is changing due to varying illumination and transitory movements, such as human activity or a swaying tree.
By learning pixel variations and adapting to slow changes over time, sudden changes in the scene can be detected. In the movie example below, a box is placed in the room. Temporary changes such as the person walking are ignored, but after the box has been present for a while it is detected as a foreground object.
MPEG demo (285 Kb)
In dynamic scenes, the task is to separate normal from abnormal behaviour. To do this, pixel phase variations are learned over time for "normal" behaviours. When a variation is observed that does not fit the model of allowable variations, it is flagged as an abnormal occurrence.
In the movie example below, the system has been trained on sequences of the subject entering the room and walking up and down. During testing, the same behaviour is ignored because it is "normal". However when the subject stops moving or jumps, that behaviour is detected as abnormal.
MPEG demo (1.3 Mb)
A major component of the project is the detection of changes in the background of scenes in which there is a lot of activity.
Fig 1. Before processing
Fig 2. The result after processing
The first photo (Fig. 1) is a still shot from a video sequence taken by a CCTV camera looking at the forecourt of terminal 2 at Heathrow airport. The image in the camera is constantly changing due to the wind moving trees, clouds moving across the sun and camera shake as well as the constantly moving traffic.
The second photo (Fig. 2) is the same shot after processing (in real time) using the algorithms being developed. We have detected that two vehicles have been parked in unexpected places. Notice that the standard background has been successfully eliminated.
- S. Gong and T. Xiang. Recognition of group activities using a dynamic
probabilistic network. In Proc. IEEE International Conference on Computer Vision,
Nice, France, October 2003.
- T. Xiang, S. Gong and D. Parkinson. Outdoor activity recognition using multi-linked
temporal processes. In Proc. British Machine Vision Conference, Norwich,
UK, September 2003.
- A. Graves and S. Gong. Spotting scene change for indexing surveillance video.
In Proc. British Machine Vision Conference, Norwich, UK, September 2003.
- T. Xiang and S. Gong. Discovering Bayesian causality among visual events in
a complex outdoor scene. In Proc. IEEE International Conference on Advanced
Video- and Signal-based Surveillance, Miami, USA, July 2003.
- S. Gong and T. Xiang. Scene event recognition without tracking. Special
issue on visual surveillance of dynamic scenes, Acta Automatica Sinica (Chinese
Journal of Automation), Chinese Academy of Sciences, Vol. 29, No. 3, pages
321-331, May 2003.
- J. Ng and S. Gong. On the binding mechanism of synchronised visual events.
In Proc. IEEE Workshop on Motion and Video Computing, Orlando, USA, December
- S. Gong and H. Buxton. Editorial:
Understanding visual behaviour. Image and Vision Computing, Vol.
20, No. 12, pages 825-826, October 2002.
- S. Gong, J. Ng and J. Sherrah. On
the semantics of visual behaviour, structured events and trajectories
of human action. Image and Vision Computing, Vol. 20, No. 12,
pages 873-888, October 2002.
- T. Xiang, S. Gong and D. Parkinson. Autonomous visual events detection and
classification without explicit object-centred segmentation and tracking.
In Proc. British Machine Vision Conference, Cardiff, September 2002.
- J. Ng and S. Gong. Learning intrinsic video content using Levenshtein distance
in graph partitioning. In Proc. European Conference on Computer Vision,
Part-IV, pages 670-684, Copenhagen, Denmark, May 2002.
- J. Ng and S. Gong. Learning
pixel-wise signal energy for understanding semantics. In Proc.
British Machine Vision Conference, Manchester, UK, September 2001.
- J. Sherrah and S. Gong. Automated
detection of localised visual events over varying temporal scales.
In Proc. European Workshop on Advanced Video-based Surveillance
Systems, Kingston, UK, September 2001.
- J. Sherrah and S. Gong. Continuous global evidence-based Bayesian Modality
Fusion for simultaneous tracking of multiple objects. In Proc. IEEE International
Conference on Computer Vision, Vancouver, Canada, 2001.
The ICONS project is funded and managed by DTI and EPSRC, and involves the following academic and industrial collaborators:
- Queen Mary, University of London (Project leader)
- Safehouse Technologies (Lead industrial partner)
- BAA plc
- British Telecom
- Heritage Protection Ltd