The result has stopped the Hammers in their tracks as they have suffered defeat for the first time in four top-flight outings under David Moyes.
Matters went from bad to worse for West Ham when Christian Atsu gave Newcastle a 3-1 lead, and even though the hosts pulled a goal back through Ayew, they were unable to complete the comeback.
The game could have not started more perfectly for Moyes's side, though, as Arnautovic scored his third goal in four league outings in the sixth minute of the encounter.
After Saivet lost the ball on the halfway line, the West Ham forward charged down the field and slotted the ball low past goalkeeper Rob Elliot.
Just four minutes later, though, Saivet made amends by scoring an equaliser from a free kick - the midfielder managed to find the back of the net from 30 yards out, but Adrian should have done better to keep the ball out.
The goalkeeper, who has been keeping Joe Hart on the substitutes' bench, got a slight touch on the ball, but it was a shot that should have been saved.
Newcastle were denied twice by the woodwork in the first half, with Dwight Gayle side-footing his shot from close range onto the crossbar, while Matt Ritchie's curling attempt after cutting inside bounced off the post.
As the clock ticked by, the pace of the game slowed down and Manuel Lanzini's absence was felt by the Hammers due to the lack of creativity in the middle of the pitch.
The East London outfit did produce a handful of chances, though, with Arnautovic turning provider to deliver a cross into the box for Antonio, who directed a header wide.
Towards the end of the first half, Ritchie rolled the ball across the box to Gayle, who squandered an opportunity as his attempt from 10 yards soared over the bar.
After the break, Arnautovic tried to make an early impact in the second half by holding off two defenders before trying to curl the ball into the top corner, but his effort dropped inches wide.
Newcastle began to pile on some pressure and it eventually paid off when Mohamed Diame scored against his former club to give the Magpies a 2-1 advantage.
Christian Atsu played a part as he nutmegged Pablo Zabaleta before feeding the ball to Diame, who hammered it home, but a matter of minutes later, the Magpies were in danger of losing their lead.
Justice was served, though, when Elliot went the right way to keep out Andre Ayew's attempt from the spot, which was not helped by the West Ham forward's stuttered run-up.
The hosts tried to get back on track, but they were dealt another blow when Newcastle scored from a counter-attack, which was launched after the Hammers failed to make the most of a corner.
Diame fed the ball to Atsu, who slotted his shot past Adrian after playing a neat one-two with Joselu, who has been roundly criticised by the Magpies' following this season.
It was a nervy end to the game for Benitez and his men as West Ham pulled a goal back thanks to Ayew, who netted home the rebound after Andy Carroll's header was punched away by Elliot.
The hosts continued to probe, but Newcastle managed to hold on and collect some much-needed points, which has pulled them out of the bottom three and into 15th place with a two-point advantage.
Moyes's side have dropped to 17th and are only one point above relegation.
West Ham United (3-4-3): Adrian; Reid, Ogbonna, Cresswell; Zabaleta, Kouyate, Obiang, Masuaku; Antonio, Arnautovic, Ayew
Newcastle United (4-4-2): Elliot; Yedlin, Lascelles, Clark, Manquillo; Ritchie, Saivet, Diame, Atsu; Gayle, Joselu