Pickford had earlier escaped a red card after conceding a penalty which he saved, but was beaten three times inside the last 25 minutes as the Magpies roared back to snatch a priceless 3-2 victory.
Marco Silva's men looked to be coasting after Dominic Calvert-Lewin and Richarlison had given them a 2-0 lead either side of Matt Ritchie's missed spot-kick, but Salomon Rondon's strike set up a grandstand finish in which Perez struck twice inside three minutes to help the Magpies claim a fifth successive home win.
The visitors started on the front foot with Lucas Digne, Bernard and Gylfi Sigurdsson causing problems down the left, and the Iceland international saw an early effort blocked at close range.
In an end-to-end encounter, Perez headed over from distance after Pickford had punched DeAndre Yedlin's dangerous cross to the edge of the box, but it was the visitors who took the lead with 18 minutes played.
Bernard's fine reverse pass put Digne into space and his cross was expertly headed home by Calvert-Lewin at the near post.
The Magpies should have been level in the 31st minute when, after Pickford had spilled Ritchie's cross, he hauled down Rondon in his pursuit of the loose ball and referee Lee Mason pointed to the spot, but did not reach for his pocket.
However, the England keeper redeemed himself by saving Ritchie's penalty, and the home side's misery took a turn for the worse within seconds when Richarlison tapped home Bernard's cross from close range with Isaac Hayden lying prone further up the field.
Pickford intervened once again four minutes before the break to deny Perez after he had adeptly controlled Fabian Schar's long ball, and reacted swiftly to repel his headed follow-up, and Mr Mason left the pitch at the break to a deafening chorus of boos.
Paul Dummett replaced Newcastle skipper Jamaal Lascelles as manager Rafael Benitez reverted to a flat back four, allowing Ritchie to push forward into a midfield role, although the newcomer was relieved to see Bernard shoot wide after Sigurdsson had intercepted Schar's ill-judged pass.
Rondon hooked a 56th-minute shot over the advancing Pickford and saw it dribble agonisingly wide of the unprotected net with the Magpies pressing, and they were back in it with 25 minutes remaining.
Perez headed the ball down to Rondon and after collecting the return pass, crossed for the Venezuela international to volley past Pickford.
Dummett looped a header over from Ritchie's 69th-minute cross with the visitors penned in, but Calvert-Lewin whistled a well-struck attempt just wide two minutes later with the temperature rising.
The game was blown wide-open with nine minutes remaining when Pickford could only parry Miguel Almiron's blistering strike and Perez steered the rebound home, and St James' Park erupted when the Spaniard stabbed home from close range three minutes later to win it.