Celimpilo Ngema scored on 39 minutes to give the home team a 1-0 home win and a shock 2-1 aggregate victory in the quarter-final.
Marumo are just two points above the relegation zone in the domestic league with three rounds remaining having won only five of 27 matches.
But a team composed almost entirely of rejects from other top-flight clubs have excelled in the second-tier African competition, winning eight and drawing one of 12 matches this season.
Even South African football followers dismissed the chances of Marumo making the semi-finals after being paired with 2020 Confederation Cup runners-up Pyramids.
But Gallants took a first-leg lead in Cairo last Sunday before conceding a late penalty from which former Premier League winger Ramadan Sobhi levelled.
The second-leg winner came when an attempted Pyramids clearance rebounded off a teammate to Ngema, who slammed the ball past goalkeeper Ahmed el Shenawy from close range.
South African Fagrie Lakay wasted a great chance to equalize for the Egyptian outfit soon after half-time, but he blazed wide with only Zimbabwean goalkeeper Washington Arubi to beat.
Marumo came under intense pressure in the closing stages, but held firm and the woodwork denied them a second goal in added time.
The South Africans now face Tanzanian outfit Young Africans for a place in the final with the first leg in Dar es Salaam on May 13 and the return match on May 20.
Meanwhile, Young Africans were held 0-0 at home by Rivers United of Nigeria, but won 2-0 on aggregate thanks to a brace from Congolese Kalala Mayele in west Africa last weekend.
The return match was temporarily halted during the first half due to a power outage at the national stadium.
South African clubs Orlando Pirates and SuperSport United have reached three Confederation Cup finals between them, but lost them all.
Young Africans, popularly known as Yanga, are the first Tanzanian side to reach the semi-finals in the African equivalent of the UEFA Europa League.
Former African champions ASEC Mimosas of the Ivory Coast scored in added time at the end of each half to defeat US Monastir of Tunisia 2-0 and qualify on aggregate by the same score line.
Franck ZouZou broke the deadlock in Bouake with his first goal of the African campaign and Pacome Zouzoua sealed success with his third in the Confederation Cup this season.
FAR Rabat, considered the strongest title contenders after an impressive run to the knockout stage, bowed out despite beating USM Alger of Algeria 3-2 in the Moroccan capital.
USM advanced 4-3 on aggregate to a last-four showdown with ASEC after building a two-goal first-leg advantage over the 2005 Confederation Cup winners.
Algerian Oussama Chita conceded an eighth-minute own goal to offer FAR hope, but Saadi Radouani equalized just four minutes later.
Cape Verde international Diney Borges gave FAR a 2-1 lead on the night after 60 minutes only for USM to level again, through Khaled Bousseliou 12 minutes from time.
Moroccan Mohamed Hrimat converted a penalty deep in added time to give the military club a hollow victory.
JOHANNESBURG - Cash-strapped South African Premiership strugglers Marumo Gallants eliminated expensively-assembled Egyptian club Pyramids from the CAF Confederation Cup on Sunday.