The purpose of this paper is to propose a protocol for distributed multi-agent optimization problem to minimize the average of objective functions of the agents in the network with satisfying constraints of each agent. The exact penalty method is applied to distributed optimization via a linear protocol, with employing two step-size parameters for the objective function and the constraint function of each agent. The proposed protocol works only with the decision variables and does not need additional variables such as dual variables. A proof of the convergence of the proposed protocol is provided as well as the boundedness under mild assumptions. The protocol is also illustrated by a numerical example.