Bilie-By-Nze, 55, is a close ally of the president and has held several ministerial jobs since 2006.
Later on Monday he announced a government composed of 45 ministers with no reshuffles in the key positions.
The decree listed no reason for the changes, which come ahead of presidential and legislative elections in mid-2023.
The statement said outgoing premier Rose Christiane Ossouka Raponda, 59, will now serve as vice president, whose role is to "assist" the head of state though it allows for no interim role as president.
A former defense minister and former mayor of the capital Libreville, she became prime minister in July 2020 and was the first woman to lead a Gabonese government.
President Ondimba in 2009 took over from his father Omar Bongo Ondimba, the central African country's ruler for 41 years, and is widely expected to run for re-election.