In the paper, we propose a novel voltage-controlled oscillator (VCO) employing defected ground structure (DGS) band stop filter (BSF). The VCO operates from 15.24 GHz to 15.86 GHz. A square shaped DGS etched at the lowest metal layer (M1) below 50-Ohm microstrip line on the top metal layer (M6) of 0.18 μm 1P6M CMOS process. The proposed square-shaped DGS resonator achieves smaller size and higher quality factor than those of a conventional spiral inductor resonator at Ku band frequency. The power dissipation of the VCO is 4 mW. The measured carrier frequency and phase noise are 15.76 GHz and-127.57 dBc/Hz (104.25 dBc/Hz) at 10 MHz (1 MHz) offset frequency, respectively. The measured figure of merit (FoM) is 182.9 dB. The fabricated chip core VCO size is only 0.084mm2 which increases figure of merit (FoMa) to 194 dB.