Human action recognition is a key technique for content-based video retrieval. Because a human motion consists of several sequential poses of the human, specifying each poses of the human motion is required for human action recognition. In this paper, the authors focus on end points and joint points of a human skeleton as interesting points obtainable from the human silhouette image of each video frame including a human motion because those points are important for specifying human poses in the human motion. This paper presents a stable and effective end points and joint points extraction method for the human body from 2D videos. The authors employ a perfect foreground object segmentation algorithm by background subtraction to obtain a moving object. Morphological and connection labeling-based algorithms are then performed on foreground objects. In addition, the paper considers the cast shadow and skeleton pruning problem which will influence the accuracy of the interesting points extraction. The experiments also show the good results of the proposed method.