This paper is to propose a method to predict the fatigue limit of the material having a defect of any shape. The 5056 aluminum alloy with Mg added has a fatigue limit, but in the case of specimens with Micro-Hole or smooth specimens, the fatigue crack is not initiated in the fatigue limit. Under these circumstances, the fatigue limit is equal to the fatigue crack initiation limit. Therefore, the following two experiments were carried out to elucidate the material property of5056 aluminum alloy adding Mg: (1) Fatigue test of notched specimens for the prediction of the fatigue limit in the area where the fatigue limit is equal to the fatigue crack initiation limit. (2) Fatigue test of Pre-Cracked specimens when a defect has sufficient stress concentration to initiate the non-propagating crack. From (1) and (2), we propose a method of predicting the fatigue limit of materials having a defect of any shape with a minimum number of fatigue tests. Furthermore, to test this method, we examined whether theoretical values and test results are equal for some specimens with Micro-Hole.