A member of the Rwandan Patriotic Front party, Eugene Barikana, 57, became a lawmaker in 2013 before resigning his position prior to the arrest.
"He is being prosecuted for possessing weapons illegally, he says that he got them while living with soldiers and forgot to return them," the Rwanda Investigation Bureau (RIB) said in a statement late Monday.
The RIB did not specify when the arrest took place or the nature of the weapons found, but local media quoted police as saying that officers searched his home where they discovered a grenade and an AK-47 rifle.
Local media said Barikana was arrested on Saturday, with the RIB statement saying he would remain in custody "while the investigation continues to reveal how he got the firearms and why he possessed them without the permission of the law."
RIB officials did not respond to calls from AFP for more details on the case.
If found guilty of the illegal possession of arms, Barikana could face a prison sentence of up to two years and a fine ranging between one and two million Rwandan francs ($770-$1,540).
A politician with a relatively low profile, Barikana has previously served as the director of cabinet in the prime minister's office and as a permanent secretary in the ministry of local government.
The Great Lakes nation has been ruled by President Paul Kagame's RPF party since the end of the 1994 genocide.
Forum