South Africa have taken the wicket of Upul Tharanga, but Sri Lanka have made steady progress by reaching 182-4 at tea.
The Proteas nabbed three wickets before lunch thanks to Dale Steyn and Jean-Paul Duminy.
Tharanga continued to lead Sri Lanka's attack, but the 29-year-old was out stumped after hitting 83 runs from 155 balls, swaying the momentum in South Africa's direction again.
However, Lahiru Thirimanne (32 not out) and Angelo Mathews (24 not out) were able to frustrate their opponents' bowlers by ending the afternoon session unscathed.
Sri Lanka trail by 273 runs.