related publications


We present face recognition schemes based on video streams: the majority decision rule and HMM maximum likelihood (ML) decision rules. PCA type of subspace feature analysis is fist applied to the face images in a video segment of fixed number of frames. Majority decision rule is then applied to PCA recognition results in the video segment. Discrete HMM (DHMM) is also applied to the single-frame recognition sequences. Continuous density HMM (CDHMM) is applied directly to the sequence of PCA feature vectors for ML decision on the video segment in a delayed decision manner. Experimental results are compared between these three schemes in terms of the number of state sand Gaussian mixtures of the HMMs. CDHMM-based decision rule achieved a 99% correct recognition rate on average. A geometric interpretation of ML in the feature subspace well explains the observed performances.



In the NOVA system, people are tracked in real-time and a nearby omni-camera in the video array is chosen to zoom into the face, as shown above.

We have collected 5 training and 4 testing face videos for each of the 5 people at different room locations and backgrounds on different omni-cameras. The patterns of face turning and expression were inhomogeneous between the training and testing sets. The NOVA system then logged at ~15fps for each person 4360 to 5890 training frames and 1880 to 3980 testing frames of single-frame face recognition and feature vectors. This set of data was used to compare the SFR schemes offline.

The performance index we have used is the overall correct percentage (OCP) which is the average of the success recognition percentages of the 5 people on one set of settings for a SFR scheme. The experiments were carried out to:

(1) Fined the optimum settings for the SFR schemes, and (2) Compare the optimum OCPs of the SFR schemes. Refer to the paper for more details.

Decision Rules Optimum OCP
Single-Frame FR 75.9%
SFR MAJ 81.7%
DMD 89.7%
CMD 99.0%


Streaming Face Recognition Result (2.3MB video)

Streaming Face Recognition Result (2.5MB video)

Face Localization Result (2.8MB video)

Streaming Face Recognition Result (182MB video)


For more information, refer to related publications.