Despite impressive fighting knocks from Usman Khawaja and Peter Nevill, Australia could not survive on day five against the Proteas, eventually bowled out for 361, 177 runs short of South Africa's target.
Kagiso Rabada led the South African attack with the ball, finishing with figures of 5-92.
Mitch Marsh and Usman Khawaja opened day five at the crease and Marsh was the first to go via a controversial LBW decision on 26.
Marsh was given not out by umpire Aleem Dar, before South Africa reviewed the decision. Despite the ball looking as though it would travel down leg side, the ball tracking deemed that the ball would go on to hit leg stump, resulting in the decision being overturned.
Fans and commentators weren't sold on Marsh's dismissal after it looked as though the DRS had botched the decision, with the system forcing Dar to overturn his original not-out call.
Khawaja was looking solid at the crease, however, and a fifth Test hundred looked on the cards, but just like David Warner in the first innings, the left-hander fell three runs short of the century mark after being trapped out front by JP Duminy.
Starc was then given out plumb lbw right before lunch, with Peter Nevill and Peter Siddle now flying the flag for Australia as they try to do anything they can to prolong the Test match.
Australia resumed from lunch at 7-263, with the run chase not a factor anymore.
However, the Aussies could not hold on and force a draw in the final session, despite a fighting unbeaten 60 from wicket keeper Peter Nevill.
The loss is Australia's first defeat in the opening Test of a summer on home soil since 1988.
The second Test begins in Hobart on Saturday.
