Developing a face monitoring robot for a desk worker

We have developed an autonomous mobile robot which monitors the face of a desk worker. The robot uses three kinds of information observed with its Kinect to search for the desk worker and adjusts its position for monitoring. The monitoring is based on incremental clustering of the faces. Our experiments revealed that not only Animation Units (AUs) features, which represent deviations from the neutral face, but also the pitch angle of the face normalized in a new way are necessary for a valid clustering under specific conditions. Our robot lost sight of a desk worker only once in experiments for 8 persons for about 50 minutes. The resulting clusters correspond to "yawning", "smiling", and "reading" for a half of the desk workers with high NMI (normalized mutual information), which is an evaluation measure often used in clustering.