Accurate background points detection for action recognition in practical video datasets

This paper treats the action recognition of moving objects such as humans in practical video-bases of unconstrained videos. For that, the motion of background image pixels, caused by the camera motion, in video frames of a scene strongly affects the accuracy. Removing such background moving influence is a challenging problem. State-of-The-Art human action recognition approaches adopted human detection to exclude background or directly use some largest motion pattern clusters as candidate background pixels. In this paper, the authors propose a new background estimation method according to background nature. This method could estimate moving background pixels or camera motion from the selected background pixels. Compared with foreground pixels, background pixels have a high diversity and small average distances to the several borders of a video frame. Those are important criteria to estimate background pixels. The proposed approach is based on the long term analysis of point motion trajectories which are more suitable for video image processing. Experimental results show that the proposed approach achieves a very competitive background extraction performance for practical video-bases.