The Saccadic Search algorithm

The Pointwise Model

The Saccadic Search strategy requires that an appearance-based model of the pattern of interest be constructed in the same way as for the common GD based algorithm. We will call such a model a pointwise model, since it is constructed using a set of filters centred at the same image point. In the case of Facial Features Detection we therefore extract a Gabor feature vector at the location of the relevant facial features in the images of the training set.

The example vectors for each facial feature are complemented with negative examples picked at random and are used to train a Support Vector Machine (SVM) classifier to recognize that feature. In our experiments, we used three classifiers to detect the left eye, the right eye and the mouth.

Detection of Features

The search starts with the retina positioned at a random location in the image. Gabor feature vectors are extracted at all the retinal points and the classifier that had been trained on the target facial feature is used to rate all the possible locations. A saccade is then performed to centre the sampling retina on the point that gave the best match. Note that, since the density of sampling points increases towards the centre of the retina, the search automatically becomes finer. Saccades are terminated when a local maximum is reached. The candidate feature thus determined is further analysed by means of an extended model.

The Extended Model

The accuracy of the saccadic part of the search is limited by the radius of the inner circle of the retina. In order to improve accuracy and to reject false matches a more accurate model of the facial features of interest is needed. This extended model can be obtained by placing the sampling retina over the facial features on the images of the training set and extracting a set of Gabor responses at each pixel. Gabor filters are arranged so that lower frequency channels are employed at the periphery of the retina, where the sampling rate is lower, while higher frequency channels are used in the fovea. Gabor responses from all the retinal points collected over the images of the training set are again used to train a Support Vector Machine classifier for each facial feature of interest.

Refining the Search

The extended model is employed to refine the localisation of a facial feature by pixelwise search. Being more discriminating than the pointwise model, it is also employed to establish the final score for each candidate feature.

Saccades on Probability

After a candidate facial feature has been detected, a saccade is performed to the expected position of the next search target based on a probabilistic model. Classifiers are switched so that the algorithms now looks for the other facial feature. A final evaluation of the sets of candidate facial features is perfomed based on the individual scores of their constituents as well as on their relative position.

Saccadic Patterns

The two images shown here are examples of saccadic patterns obtained when looking for the eyes of a person.

Numbers denote the places where saccades were started (saccadic patterns that do not appear to converge to any meaningful location are automatically abandoned).

In this case, after a first unsuccessful attempt was abandoned (1), saccades converged to the subject's left eye (2). A saccade was then performed to the most probable position of the other eye, thus allowing its detection (3).

Information from the outline of the orbit makes eye detection possible the even in the case the subject's eyes are shut.

Follow this link for more results on Eyes and Mouth Detection, or click here for the Saccadic Search Home Page.

Back to F. Smeraldi's Home Page
Last modified Feb 11th, 1999