Rwasa was deposed as the head of the suspended National Freedom Council (CNL) at a general assembly in the north of the country, party sources and witnesses said.
The 60-year-old former militia leader was replaced by Nestor Girukwishakae, a senior executive at a state-owned company.
Formed in 2019, CNL was suspended last year by the government of the central African country which accused it of "irregularities."
Critics said the move by the interior ministry was an attempt to stifle dissent ahead of the 2025 legislative elections and risked returning the country to political turmoil.
The party's secretary general Simon Bizimungu denounced Rwasa's removal, saying it went against their constitution.
"This is a violation of article 47 of our party's statutes, which states that only the president and legitimate legal representative is authorized to convene such a meeting," Bizimungu told AFP.
Bizimungu accused the interior minister Martin Niteretse of refusing to allow Rwasa to hold a general assembly of his own on March 2, but "gave authorization to a small group of 10 dissidents to organize this masquerade and steal our party from us."
A senior CNL official said the dissident legislators must submit the decisions of Sunday's meeting to the interior ministry for approval.
"This is the only way they have found to keep our leader Agathon Rwasa out of the legislative elections scheduled for May 2025, because they are afraid of the political force he represents today," said Bizimungu.
Local media reported that Sunday's meeting took place under a heavy police presence.
Rwasa came second to President Evariste Ndayishimiye in the 2020 election, which the opposition said was flawed.
Ndayishimiye, who took power after the death of president Pierre Nkurunziza, has been lauded by the international community for slowly ending years of isolation under his predecessor's chaotic and bloody rule.
But he has failed to improve a wretched human rights record and the country of 12 million people remains one of the poorest on the planet.