Although titanium alloy compacts manufactured by metal injection molding (MIM) process have the same static mechanical properties as that of wrought materials with similar composition, their fatigue strength shows remarkably lower levels. In this study, improvement of the fatigue strength of injection molded Ti-6Al-4V alloy compacts was performed by refining the pore and grain sizes through different approaches such as the usage of fine powders, dual phase region sintering, and hydrogenation and dehydrogenation treatment. Especially, the effects of pore size and grain size on the fatigue strength were evaluated. Essentially, the relationship between the fatigue strength and the grain size for HIP treated compacts obeyed the Hall-Petch law. On the other hand, for the injection molded compacts with pores, it was found that the largest pore dominates the fatigue strength in the case of the large pore diameter ratio to the grain diameter, while the relative density influences on the fatigue strength when the pore diameter ratio is small.