South Africa was cruising to a historic first win in five matches at the World Cup after goals by Linda Motlhalo and Thembi Kgatlana either side of half-time in Dunedin, New Zealand.
But Argentina pulled one back via a stunning Sophia Braun strike in the 74th minute and then drew level five minutes later when Romina Nunez headed home.
South Africa also threw away the lead, and then conceded the decisive goal in the last minute, to lose 2-1 to Sweden in their opening match.
"It is about taking our chances, making better decisions in the final third. If we take our chances we have a different conversation," coach Desiree Ellis said.
"Today was a great opportunity to put ourselves a step ahead, but it is not all lost."
Appearing at their fourth World Cup, Argentina have still never won in 11 attempts themselves.
"I am really proud of how we came back, we never gave up. We showed a lot of heart," said Braun, Argentina's U.S.-born full-back.
After losing their opening matches in New Zealand, Argentina and South Africa have one point each in Group G.
Favorites Sweden and Italy have three points before playing each other in Wellington on Saturday, before a decisive final round of games on August 2.
Also on Saturday, France faces Brazil in Brisbane in what promises to be a cracker, while Jamaica play Panama in the same Group F.
Jamaica is another team chasing a first win at any World Cup and defender Allyson Swaby said doing so would send a message to the country's football bosses that the women's team deserve better treatment.
England win
England stood on the brink of the Women's World Cup last 16 after a Lauren James cracker in the sixth minute gave them a 1-0 win over Denmark.
But an otherwise hugely satisfactory night for England in front of just over 40,000 fans in Sydney was marred by what looked like a serious knee injury to influential midfielder Keira Walsh.
On a night of mixed emotions, Sarina Wiegman's side will seal their place in the knockout rounds if Asian champions China fail to beat debutants Haiti later Friday.
Denmark might have stolen a point with three minutes left in normal time when, despite having as little as 20% of the possession for much of the game, Amalie Vangsgaard shaved the outside of the England post with a header.
Wiegman made two changes from the team that squeezed past Haiti 1-0 in their opener, Rachel Daly and James coming into the starting XI.
And it was the 21-year-old Chelsea forward James who was the star of the first half, before fading in the second.
England's Lauren James, right, celebrates a first half goal with teammate England's Ella Toone during the Women's World Cup Group D soccer match between England and Denmark at Sydney Football Stadium in Sydney, Australia, Friday, July 28, 2023.
"It's an amazing feeling and something I always dreamed of," she said. "We built on the momentum from the last win and took it into this game. Another difficult win, but we got the win and that is the most important thing."
In the final round of Group D games on Tuesday, England faces China, and Denmark will play Haiti.