We investigate the magnetization dynamics induced by a current pulse in Permalloy nanowires by means of Lorentz microscopy and electron holography, together with simultaneous transport measurements. A variety of magnetization dynamics is observed below the Curie temperature. Local transformation, displacement of magnetic domain wall and nucleation and annihilation of magnetic domain, i.e. magnetization reversal are presented as a function of current density flowing into the wire and wire resistance. Shift of threshold current densities for domain wall displacement and magnetization reversal when changing current pulse duration and thermal conductance of the sample supports that observed behavior of magnetic domains and domain walls is associated with the spin transfer torque and thermal excitation. For the well-controlled magnetization reversal, we microscopically demonstrate that applying small in-plane magnetic field is very effective to controllably nucleate and erase the magnetic domain using a current pulse. Stochastic nature of the magnetization reversal due to spin-wave and thermal excitation in the absence of magnetic field completely disappears and turns into deterministic in the presence of small magnetic field, which enables the magnetization reversal control using current.