Underwater manipulators are traditionally controlled by skilled operators to achieve tasks, e.g. underwater survey, inspection, construction and repair operations. However, these systems need much human resources and huge control systems. Thereby, the automation of control of underwater manipulators are required to reduce the work load of operators. In order to satisfy the requirements, this paper introduces a method that controls underwater manipulators using visual information observed by stereo cameras. The characteristics of the method is the robustness against calibration errors of cameras and kinematical relation between cameras and manipulators. Operators can control manipulators adaptively by pointing a target on images captured by cameras using the method. In this paper, the usefulness of the method is verified through experimental results using an underwater robotic arm which has 4 DOFs.