BEWARE: Behaviour based Enhancement of Wide-Area Situational Awareness in a Distributed Network of CCTV Cameras


 

BEWARE is a project funded by EPSRC and MOD to develop models for video based object space-time association (consistent labelling) and behaviour interpretation across a distributed network of CCTV cameras for the enhancement of global situational awareness in a wide area. 

 

A distributed network of multi-camera system is capable of capturing time-synchronised visual data over large physical spaces, providing the potential for gaining a global understanding of object activities (behaviour analysis) and object whereabouts (association) across different spaces and over time. However, capturing more data does not necessarily provide automatically better information for object behaviour interpretation and space-time association. Such a system can also cause overflow of data and confusion if data content is not analysed effectively in order to extract the most relevant and timely information. Moreover, current CCTV cameras are mostly controlled manually by operators based on ad hoc criteria. The BEWARE project aims to develop automated computer vision systems to model and interpret cooperatively object behaviours in public spaces (in particular people and vehicles) across a distributed network of cameras for effective global anomaly detection and object association/tracking across disjoint spaces over time. Specifically, we focus on:

1.    Developing models for robust detection and association of people over wide areas of different physical sites captured by a distributed network of cameras, known as the re-identification problem, e.g. monitoring the activities of a person travelling through different locations in a public transport infrastructure environment such as an airport or an underground station.

2.    Developing models for global situational awareness enhancement by learning to correlate object local behaviours observed at each camera viewpoint across a network of disjoint cameras located at different physical sites, and for the detection of abnormal behaviours in public space across camera views.

3.    Developing models for learning visual context directly from limited observations without operator manual labelling, in order to provide a mechanism for coping with changes in behavioural context and definitions of anomaly. Effective and robust abnormal behaviour recognition needs be adaptive to changes in time, location, and the context of visual environment.

4.    Developing models for incorporating minimal human feedback to enhance behaviour model learning under limited/incomplete visual observations, e.g. modelling rare behaviours and discovering new classes of unknown behaviours of significance in public spaces.

5.    Developing models for robust human action centred behaviour recognition in an unconstrained and crowded public environment.


Publications

  1. T. Hospedales, S. Gong and T. Xiang. Video Behaviour Mining Using a Dynamic Topic Model. International Journal of Computer Vision, in print, 2012.
  2. C.C. Loy, T. Xiang and S. Gong. Incremental Activity Modelling in Multiple Disjoint Cameras. IEEE Transactions on Pattern Analysis and Machine Intelligence, in print, 2012.
  3. J. Li, S. Gong and T. Xiang. Learning Behavioural Context. International Journal of Computer Vision, in print, 2012.
  4. T. Hospedales, S. Gong and T. Xiang. Finding Rare Classes: Active Learning with Generative and Discriminative Models. IEEE Transactions on Knowledge and Data Engineering, in print, 2012.
  5. W. Zheng, S. Gong and T. Xiang. Transfer Re-identification: From Person to Set-based Verification. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, Providence, Rhode Island, USA, June 2012.
  6. C.C. Loy, T. Hospedales, T. Xiang and S. Gong. Stream-based Joint Exploration-Exploitation Active Learning. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, Providence, Rhode Island, USA, June 2012.
  7. C.C. Loy, T. Xiang and S. Gong. Salient Motion Detection in Crowded Scenes. In Proc. 5th International Symposium on Communications, Control and Signal Processing, Rome, Italy, May 2012.
  8. W. Zheng, S. Gong and T. Xiang. Quantifying and Transferring Contextual Information in Object Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 34, No. 4, pp. 762-777, April 2012.
  9. M. Bregonzio, T. Xiang and S. Gong. Fusing Appearance and Distribution Information of Interest Points for Action Recognition. Pattern Recognition, Vol. 45, No. 3, pp. 1220-1234, March 2012.
  10. C. Shan, F. Porikli, T. Xiang and S. Gong (Eds.). Video Analytics for Business Intelligence, 368 pages, Springer, March 2012.
  11. S. Gong and T. Xiang. Visual Analysis of Behaviour: From Pixels to Semantics, 376 pages, Springer, 2011 (download preface, table of contents, introduction and index).
  12. T. Hospedales, S. Gong and T. Xiang. Learning Tags from Unsegmented Videos of Multiple Human Actions. In Proc. IEEE International Conference on Data Mining, Vancouver, Canada, December 2011.
  13. S. Gong, C.C. Loy and T. Xiang. Security and Surveillance. In Moeslund, Hilton, Kruger and Sigal (Eds.), Visual Analysis of Humans: Looking at People, pp. 455-472, Springer, September 2011.
  14. T. Hospedales, J. Li, S. Gong and T. Xiang. Identifying Rare and Subtle Behaviours: A Weakly Supervised Joint Topic Model. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 33, No. 12, pp. 2451-2464, December 2011.
  15. C.C. Loy, T. Xiang and S. Gong. Detecting and Discriminating Behavioural Anomalies. Pattern Recognition, Vol. 44, No. 1, pp. 117-132, January 2011.
  16. J. Li, T. Hospedales, S. Gong and T. Xiang. Learning Rare Behaviours. In Proc. Asian Conference on Computer Vision, Queenstown, New Zealand, November 2010.
  17. C.C. Loy, T. Xiang and S. Gong. Stream-based Active Anomaly Detection. In Proc. Asian Conference on Computer Vision, Queenstown, New Zealand, November 2010.
  18. C.C. Loy, T. Xiang and S. Gong. Time-Delayed Correlation Analysis for Multi-Camera Activity Understanding. International Journal of Computer Vision, Vol. 90, No. 1, pp. 106-129, October 2010.
  19. S. Gong, T. Xiang and S. Hongeng. Learning Human Pose in Crowd. In Proc. ACM International Conference on Multimedia, Workshop on Multimodal Pervasive Video Analysis, Firenze, Italy, October 2010.
  20. B. Prosser, W. Zheng, S. Gong and T. Xiang. Person Re-Identification by Support Vector Ranking. In Proc. British Machine Vision Conference, Aberystwyth, Wales, September 2010.
  21. M. Bregonzio, J. Li, S. Gong and T. Xiang. Discriminative Topics Modelling for Action Feature Selection and Recognition. In Proc. British Machine Vision Conference, Aberystwyth, Wales, September 2010.
  22. M. Bregonzio, S. Gong and T. Xiang. Action Recognition with Cascaded Feature Selection and Classification. In Proc. IEE International Conference on Imaging for Crime Detection and Prevention, London, UK, December 2009.
  23. C.C. Loy, T. Xiang and S. Gong. Modelling Activity Global Temporal Dependencies using Time Delayed Probabilistic Graphical Model. In Proc. International Conference on Computer Vision, Kyoto, Japan, October 2009.
  24. J. Li, S. Gong and T. Xiang. Discovering Multi-Camera Behaviour Correlations for On-the-Fly Global Activity Prediction and Anomaly Detection. In Proc. International Workshop on Visual Surveillance, Kyoto, Japan, October 2009.
  25. C.C. Loy, T. Xiang and S. Gong. Multi-Object Activity Modelling Using Gaussian Processes. In Proc. British Machine Vision Conference, London, September 2009.
  26. E. Zelniker, T. Hospedales, S. Gong and T. Xiang. A Unified Approach for Adaptive Multiple Feature Tracking for Surveillance Applications. In Proc. British Machine Vision Conference, London, September 2009.
  27. M. Bregonzio, S. Gong and T. Xiang. Recognising Action as Clouds of Space-Time Interest Points. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, Miami, USA, June 2009.
  28. C.C. Loy, T. Xiang and S. Gong. Multi-Camera Activity Correlation Analysis. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, Miami, USA, June 2009.
  29. J. Li, S. Gong and T. Xiang. Scene Segmentation for Behaviour Correlation. In Proc. European Conference on Computer Vision, Marseille, France, October 2008.
  30. B. Prosser, S. Gong and T. Xiang. Multi-camera Matching under Illumination Change Over Time. In Proc. ECCV Workshop on Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications, Marseille, France, October 2008.
  31. C.C. Loy, T. Xiang and S. Gong. From Local Temporal Correlation to Global Anomaly Detection. In Proc. International Workshop on Machine Learning for Vision-based Motion Analysis, Marseille, France, October 2008.
  32. E. Zelniker, S. Gong and T. Xiang. Global Abnormal Behaviour Detection Using a Network of CCTV Cameras. In Proc. International Workshop on Visual Surveillance, Marseille, France, October 2008.
  33. J. Li, S. Gong and T. Xiang. Global Behaviour Inference Using Probabilistic Latent Semantic Analysis. In Proc. British Machine Vision Conference, Leeds, September 2008.
  34. B. Prosser, S. Gong and T. Xiang. Multi-camera Matching Using Bi-Directional Cumulative Brightness Transfer Functions. In Proc. British Machine Vision Conference, Leeds, September 2008.

Partners

BEWARE involves the following academic, governmental and industrial partners: