Several studies have been provided the principal activated regions and locations during action observation. However, it remains unclear that the dynamic link and information flows in different visual perspectives during the action observation tasks. The aim of this study was investigating the effective connectivity of different visual perspectives during action observation tasks and classify the perspectives. Eleven regions were selected as the regions of interest (ROI) by meta-analysis using the term of "action observation"for this study. We employed weighted and directed (WD) brain networks by Generalized partial directed coherence (GPDC) on the fMRI BOLD time series of action observation tasks from different visual perspectives. Afterward, the paired sample t-test was implemented to find the alteration of brain networks between the two perspectives. According to the results, global efficiency, local efficiency and cluster coefficient were statistically significantly different between first-person perspective (1PP) and third-person perspective (3PP) in Supplementary motor cortex (SMA) and Right inferior parietal lobe (RIPL) and Left inferior parietal lobe (LIPL). These three regions which we considered may provide the additional support for the visuospatial transformation from 3PP to 1PP. WD brain network topological measures of SMA, RIPL and LIPL were also selected to classify the 1PP and 3PP using support vector machine (SVM) with the best accuracy close to 90%.