Final results showed that 20 out of 26 cantons were against while 56.3 percent of the population rejected the initiative, AFP reported on Sunday.
The proposal was introduced by an alliance of non-governmental groups and center-left parties in order to have rifles stored in armories instead of keeping them at residential areas.
In order for the referendum to be passed both a majority of districts and a majority of people should have voted in favor of the move.
The results have dashed hopes of anti-gun campaigners, including women, doctors and police officers, who argue that the easy availability of arms have largely contributed to the high rates of domestic gun violence, suicides and homicide in the Alpine country.
Official figures show that between 1.2 million and 4.5 million rifles are estimated to be in circulation in Switzerland, putting the country behind only the US and Yemen in guns per capita.
The government has voiced opposition to the initiative, arguing that the "current legislation assures adequate and sufficient protection of the population against the abusive use of weapons."
Meanwhile, Swiss Justice Minister Simonetta Sommaruga stated that in spite of the failure of the proposal to stiffen the gun control, Switzerland would remain a safe country.