The ferry, on an overnight trip from the capital Libreville to the city of Port-Gentil, reported "water intrusion" between 0200 GMT and 03:00GMT, the vessel's owner, Royal Cost Marine, said on Facebook.
"This intrusion led to loss of control over the boat. The authorities have been alerted, emergency teams are at the site and the passengers are currently being rescued," it said.
The vessel had departed with 151 passengers on board, the port source said, speaking on condition of anonymity.
Six ambulances were waiting alongside a fire truck on a jetty in Port Mole, the vessel's departure point, and motorized canoes had started bringing in survivors, an AFP reporter saw.
A young woman was taken to an ambulance on a stretcher, covered in a survival blanket.
Other survivors disembarked from an oil company's supplies barge, which had taken part in the rescue operation, as a crowd applauded.
Prime Minister Alain-Claude Bilie-By-Nze and Interior Minister Lambert Noel-Matha were at the port.
Fatal Ferry Accident in Gabon
LIBREVILLE - At least two people have died and 28 are missing after a small ferry foundered off the Gabon coast, a port official told AFP on Thursday.