INSIGHT: Video Analysis and Selective Zooming using Semantic Models of Human Presence and Activity


INSIGHT is a project funded by EPSRC and MOD DSTL under the EPSRC Technologies for Crime Prevention and Detection Programme. INSIGHT aims to advance techniques for semantic content analysis of CCTV recordings for automatic semantic video tagging, search and pro-active sampling by:  

 

(1) Developing models for fully automated semantic-tagging of CCTV recordings based on holistic human presence detection and abnormal event / activity recognition, e.g. monitoring unmanned sites and buildings and to significantly reduce the false alarms triggered by existing Video Motion Detection systems.

(2) Developing models for event and activity based visual topic spotting and scene change detection for semantic decomposition and automatic sorting of CCTV recordings over time, e.g. automatically detecting in video aggressive human behaviour on buses, trains or in front of buildings.

(3) Developing models for automated selective zooming and super-resolution in CCTV recordings with variable levels of details, e.g. to synthesize in arbitrary virtual views good-quality close-up images of a face or vehicle number-plate in order to improve the accuracy of automatic face-recognition and ANPR (Automatic Number Plate Recognition), and to increase the value of imagery evidence captured in low-resolution by CCTV cameras.


Publications

1.    J. Zhang and S. Gong. Action Categorisation by Structural Probabilistic Latent Semantic Analysis. Computer Vision and Image Understanding, Vol. 114, No. 8, pp. 857-864, August 2010.

2.    J. Zhang and S. Gong. Action Categorisation with Modified Hidden Conditional Random Field. Pattern Recognition, Vol. 43, No. 1, pp. 197-203, January 2010.

3.    J. Zhang and S. Gong. People Detection in Low-Resolution Video with Non-Stationary Background. Image and Vision Computing, Vol. 27, No. 4, pp. 437-443, March 2009.

4.    T. Xiang and S. Gong. Optimising Dynamic Graphical Models for Video Content Analysis. Computer Vision and Image Understanding, Vol. 112, No. 3, pp. 310-323, December 2008.

5.    T. Xiang and S. Gong. Incremental and Adaptive Abnormal Behaviour Detection. Computer Vision and Image Understanding, Vol. 111, No. 1, pp. 59-73, July 2008.

6.    D. Russell and S. Gong. Multi-Layered Decomposition of Recurrent Scenes. In Proc. European Conference on Computer Vision, Marseille, France, October 2008.

7.    D. Russell and S. Gong. Exploiting Periodicity in Recurrent Scenes. In Proc. British Machine Vision Conference, Leeds, September 2008.

8.    T. Xiang and S. Gong. Activity based Surveillance Video Content Modelling. Pattern Recognition, Vol. 41, No. 7, pp. 2309-2326, July 2008.

9.    K. Jia and S. Gong. Generalised Face Super-Resolution. IEEE Transactions on Image Processing, Vol. 17, No. 6, pp. 873-886, June 2008.

10.  T. Xiang and S. Gong. Video Behaviour Profiling for Anomaly Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 30, No. 5, pp. 893-908, May 2008.

11.  T. Xiang and S. Gong. Spectral Clustering with Eigenvector Selection. Pattern Recognition, Vol. 41, No. 3, pp. 1012-1029, March 2008.

12.  S. Gong, C. Shan and T. Xiang. Visual Inference of Human Emotion and Behaviour. In Proc. ACM International Conference on Multimodal Interfaces, invited paper, pp. 22-29, Nagoya, Japan, November 2007.

13.  D. Russell and S. Gong. Segmenting highly textured nonstationary background. In Proc. British Machine Vision Conference, Warwick, September 2007.

14.  S. Gong. Holistic video detection. In Proc. SPIE Conference on Optics and Photonics for Counter-Terrorism and Crime-Fighting, invited paper, Florence, Italy, September 2007.

15.  T. Xiang and S. Gong. Model selection for unsupervised learning of visual context. International Journal of Computer Vision, Vol. 69, No. 2, pp. 181-201, 2006.

16.  T. Xiang and S. Gong. Beyond tracking: Modelling activity and understanding behaviour. International Journal of Computer Vision, Vol. 67, No. 1,  pp. 21-51, 2006.

17.  K. Jia and S. Gong. Hallucinating multiple occluded face images of different resolutions. Pattern Recognition Letters, Vol. 27, No. 15, pp. 1768-1775, November 2006.

18.  K. Jia and S. Gong. Multi-resolution patch tensor for facial expression hallucination. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, Vol. 1, pp. 395-402, New York, June 2006.

19.  K. Jia, S. Gong and A. Leung. Coupling face registration and super-resolution. In Proc. British Machine Vision Conference, Vol. 2, pp. 449-458, Edinburgh, September 2006.

20.  T. Xiang and S. Gong. Optimal dynamic graphs for video content analysis. In Proc. British Machine Vision Conference, Vol. 1, pp. 177-186, Edinburgh, September 2006.

21.  Y. Raja and S. Gong. Sparse multiscale local binary patterns. In Proc. British Machine Vision Conference, Vol. 2, pp. 799-808, Edinburgh, September 2006.

22.  D. Russell and S. Gong. Minimum Cuts of A Time-Varying Background. In Proc. British Machine Vision Conference, Vol. 2, pp. 809-818, Edinburgh, September 2006.

23.  T. Xiang and S. Gong. Incremental visual behaviour modelling. IEEE Visual Surveillance Workshop, Graz, May 2006.

24.  J. Zhang and S. Gong. Beyond static detectors: A Bayesian approach to fusing long-term motion with appearance for robust people detection in highly cluttered scenes. IEEE Visual Surveillance Workshop, Graz, May 2006.

25.  H. Hung and S. Gong. Classification of human interaction from a distance using salient body behaviour modelling. SPIE Conference on Optics and Photonics for Counter-Terrorism and Crime-Fighting, Stockholm, September 2006.

26.  T. Xiang and S. Gong. Visual learning given sparse data of unknown complexity. IEEE International Conference on Computer Vision, Beijing, October 2005.

27.  K. Jia and S. Gong. Multi-modal tensor face for simultaneous super-resolution and recognition. IEEE International Conference on Computer Vision, Beijing, October 2005.

28.  T. Xiang and S. Gong. Video behaviour profiling and abnormality detection without manual labelling. IEEE International Conference on Computer Vision, Beijing, October 2005.

29.  T. Xiang and S. Gong. Online video behaviour abnormality detection using reliability measure. British Machine Vision Conference, Oxford, September 2005.

30.  K. Jia and S. Gong. Multi-modal face image super-resolutions in tensor space. British Machine Vision Conference, Oxford, September 2005.

31.  T. Xiang and S. Gong. Relevance learning for spectral clustering with applications on image segmentation and video behaviour profiling. IEEE International Conference on Advanced Video and Signal based Surveillance, Como, September 2005.

32.  D. Russell and S. Gong. A highly efficient block-based dynamic background model. IEEE International Conference on Advanced Video and Signal based Surveillance, Como, September 2005.

33.  K. Jia and S. Gong. Hallucinating multiple occluded CCTV face images of different resolutions. IEEE International Conference on Advanced Video and Signal based Surveillance, Como, September 2005.

34.  H. Hung and S. Gong. Detecting and quantifying unusual interactions by correlating salient motion. IEEE International Conference on Advanced Video and Signal based Surveillance, Como, September 2005.

35.  K. Jia and S. Gong. CCTV face hallucination under occlusion with motion blur. IEE International Symposium on Imaging for Crime Detection and Prevention, London, UK, June 2005.

36.  T. Xiang and S. Gong. Unsupervised learning of visual context using mixture models. IEE International Conference on Visual Information Engineering, Glasgow, April 2005.

37.  A. Graves and S. Gong. Surveillance video indexing with iconic patterns of activity. IEE International Conference on Visual Information Engineering, Glasgow, April 2005.

38.  S. Gong. Finding a needle in haystacks: Towards behaviour recognition based video surveillance, Security and Defence 2004, invited article, London, October 2004.

39.  A. Graves and S. Gong. Wavelet-based holistic sequence descriptor for generating video summaries. British Machine Vision Conference, pp. 167-176, Kingston-upon-Thames, England, September 2004.

40.  T. Xiang and S. Gong. Activity based video content trajectory representation and segmentation. British Machine Vision Conference, Kingston-upon-Thames, England, September 2004.

41.  H. Hung and S. Gong. Quantifying temporal saliency. British Machine Vision Conference, Kingston-upon-Thames, England, September 2004.


Partners

INSIGHT involves the following academic, government and industrial collaborators:


Links