In this paper, for the purpose of monitoring people moving on the ground from a low-altitude aerial vehicle, we propose a method for stable image stabilization using planar information of the ground. In existing methods the homography-based method has been popularly used for image stabilization. However, in case that captured images consist of non-planar areas, the performance of the homography-based method decreases. The essential-based method works well in this scene, but the accuracy gets worse in case that images mainly include planar areas. Thus in the proposed method, we utilize the Geometric Robust Information Criterion (GRIC) model to choose a method from the homography-based method or the essential-based method. After the selection of the method, we extract a planar area in the scene by fitting a plane using RANSAC, followed by detection of people on the ground with high accuracy. Experimental results confirm that the effectiveness of the proposed method.