Much of the first half was played in an open fashion, but it was Freiburg who created the better chances as Admir Mehmedi forced Lukas Kruse into a fine save with a powerful effort.
Kruse played a key role in the fightback as he produced a crucial stop to keep out a Pavel Krmas header early in the second half.
The save proved even more important when Paderborn equalised with 20 minutes remaining. Srdan Lakic squared the ball for Rupp, who had the easy task of tapping in from a few yards out.
Paderborn completed the turnaround 10 minutes later as Rupp's clever movement off the ball allowed him to slot in the winner from a Mahir Saglik cross.
An extended spell of pressure almost earned Freiburg a late equaliser, but Christian Gunter could only hit the post late on as Paderborn moved a point clear of the hosts, who are level on points with 16th-placed Hannover.