During the Winter War (1939–1940), between Finland and the Soviet Union, he began his duty as a sniper and fought for the Finnish Army against the Red Army. In temperatures between −40 and −20 degrees Celsius, dressed completely in white camouflage, Häyhä was credited with 505 confirmed kills of Soviet soldiers.[2][4] A daily account of the kills at Kollaa was conducted for the Finnish snipers. Remarkably, all of Häyhä's kills were accomplished in fewer than 100 days – in other words, an average of 5 kills per day – at a time of year with very short hours of daylight.[5][6][7]

Häyhä used a Finnish militia variant of the Russian-made Mosin-Nagant rifle, the White Guard M/28 "Pystykorva" (literally Spitz, due to the sight's resemblance), because it suited his small frame (5 ft 3 in/1.60 m). He preferred to use iron sights rather than telescopic sights to present a smaller target (the sniper must raise his head higher when using a telescopic sight), for reliability (a telescopic sight's glass can fog up easily in cold weather) and for aid in concealment (sunlight glare in telescopic sight lenses can reveal a sniper's position).

The Soviets tried several ploys to get rid of him, including counter-snipers and artillery strikes. On March 6, 1940, Häyhä was shot in the lower left jaw by a Russian soldier during combat. The bullet tumbled upon impact and exited his head. He was picked up by fellow soldiers who said "half his head was missing", but he was not dead: he regained consciousness on March 13, the day peace was declared. Shortly after the war, Häyhä was promoted from Alikersantti (Corporal) to Vänrikki (Second Lieutenant) by Field Marshal Carl Gustaf Emil Mannerheim. No-one else has gained rank so quickly in Finland's military history.