The influence of color on the perception of loudness and annoyance caused by noise were measured using a magnitude estimation method. In the first experiment, ten chromatic colors (red, yellow-red, yellow, green-yellow, green, blue-green, blue, purple-blue, purple, red-purple) and two achromatic colors (gray, white) were used as the visual component of an audio-visual stimulus. The choice of chromatic colors was based on the Munsell color system. The auditory component of the stimulus was a wide range of pink noise (40 dB to 90 dB by 12.5 dB steps). The visual component projected on a full screen via a liquid-crystal projector. The auditory component presented through a loudspeaker. The results showed that color significantly influenced the perception of loudness and annoyance associated with noise. When the color was close to red or white, the noise was perceived as significantly louder and noisier. When the color was close to green or blue, the noise was perceived as significantly softer and quieter. In the second experiment, the effect of chroma of color on the perceived loudness caused by noise was systematically investigated. In this experiment, chroma of color was manipulated while hue and value was kept constant. The auditory stimulus was a narrow range of pink noise (61 dB to 69 dB by 2 dB steps). The results revealed that the loudness of noise increased when the chroma of color was higher and decreased when the chroma was lower.