Tunisia then completed their comeback in the 66th minute through Khazri, who was on hand to tap Oussama Haddadi's cutback into the back of the Panama net.
With neither side able to qualify for the last 16 of the competition, the pressure was off at the Mordovia Arena, and both nations made numerous changes to their respective starting XIs.
Although Tunisia were on top from the off, dominating possession and showing more cohesion, Panama were still capable of fashioning dangerous chances on the counter.
Indeed, Los Canaleros went on to take the lead against the run of play as Roman Torres's blocked effort fell to Rodriguez, whose ensuing drive took a huge deflection off Meriah to leave goalkeeper Aymen Mathlouthi wrongfooted.
Panama were able to hold on to their lead until half time, despite being overrun in terms of possession, thanks to Tunisia's toothlessness in front of goal, but after a slow start to the second half, the Carthage Eagles cranked into gear and eventually drew level.
A neat passing move initiated by Naim Sliti saw Khazri slide a low ball across the face of goal for Ben Youssef to lift into the net from close range, which not only got Tunisia back into the game, but also made history as the World Cup's 2,500th goal.
Goal number 2,501 would not be far away, Khazri going from provider to scorer in a similarly incisive manner to their first goal, Sliti and Haddadi exchanging passes down the left before the latter teed up the Sunderland man for an easy tap-in.
Tunisia were able to see the game out, though there were some nervy moments as Panama probed forward in search of an equaliser for their first point, and even though both teams will be leaving Russia soon, both can be pleased with their accomplishments throughout the tournament.
While the Carthage Eagles recorded their first win at the finals since their 3-1 triumph over Mexico in 1978, Panama gave a good account of themselves at their debut tournament and scored not once but twice in their three games.
PANAMA (4-3-3): Penedo; Machado, Roman Torres (Tejada 56'), Escobar, Ovalle; Barcenas, Godoy, Gomez; Avila (Arroyo 81'), Rodriguez, Gabriel Torres (Cummings 45')
TUNISIA (4-1-4-1): Mathlouthi; Nagguez, Bedoui, Meriah, Haddadi; Sassi (Badri 45'); Skhiri, Chaalali, Ben Youssef; Khazri (Srarfi 89'); Sliti (Khalil 77')