In front of six African presidents in the crowd of 50,000, the 49-year-old party organizer-turned-politician vowed to act with "integrity" to lift the country of 29 million out of its poverty.
"Madagascar is today under construction. The transformation is underway, rising on the horizon," he said before watching a military parade.
But the absence of two former presidents, Marc Ravalomanana and Hery Rajaonarimampianina, who attended Rajoelina's last inauguration in 2018, highlighted early tensions facing the new administration.
They are part of an opposition coalition that held near-daily protests for weeks before the November 16 vote to denounce what they called a "constitutional coup d'etat" by the president to stay in office.
Demonstrations were banned in the capital, Antananarivo, for five days ahead of Saturday's ceremony.
Most opposition candidates boycotted the election and the coalition has insisted it will not recognize the result. Rajoelina secured victory without needing a second-round vote, though only 46% of eligible voters cast ballots.
Rajoelina, who became president a first time without an election in 2009 after Ravalomanana was ousted in a military mutiny, has quickly faced international pressure.
'Reforms' needed
After the election, ambassadors from the European Union, United States and other major donor countries expressed concern at "tensions and incidents" that marked the campaign.
The U.S. State Department said that "domestic and international election observers, noted irregularities in the electoral process."
Western nations said in a joint statement that Rajoelina must "take steps to restore confidence conducive to dialogue" and carry out legal and electoral reforms ahead of future votes.
The economy is in crisis and Rajoelina has said he will devote his new term to boosting the livelihoods in what is one of the world's poorest countries, despite its natural wealth.
The resource-rich island country is the world's biggest producer of vanilla but a price fall amid sluggish world demand has hit the industry and Madagascar's balance of payments.
About 75% of the population live below the poverty level.
The president has visited poor parts of the capital in recent days handing out solar lamps in areas that have no electricity.
His party has condemned the opposition as "irresponsible" though Rajoelina has said he will work with "constructive" elements.
Some opposition activists have faced criminal charges for taking part in the demonstrations before the election that were often broken up with tear gas.
Ahead of the inauguration, Rina Randriamasinoro, secretary general of the TIM opposition party, was jailed for two years for taking part in a banned demonstration.
Forum