The Gunners will travel to Bournemouth at the end of the month after England Under-21 winger Nelson finished from close-range to settle a tie which saw Sky Bet Championship leaders Leeds more than match their top-flight opponents.
A decade on from a memorable FA Cup shock at rivals Manchester United, Leeds looked set to once again ruffle the feathers of the established order, only to fall agonisingly short after a fine display in front of a vast away following at the Emirates Stadium.
Arsenal head coach Mikel Arteta, taking charge of just his fourth match, must have displayed the half-time attributes of someone like his opposite number Marcelo Bielsa – whose first foray into management came when his counterpart was just eight years old – as Arsenal were a different prospect after a dismal first-half.
Leeds wasted a glut of chances to take the lead and make it a jittery night for Arteta and the home fans, Patrick Bamford coming closest when he rattled the frame of the goal.
But, after the break, Arsenal responded to whatever Arteta demanded of them and Nelson's tap-in on 55 minutes was enough to seal a hard-fought victory.
But it was the second-tier side who were using the ball better in the opening stages, working it into dangerous positions without finding an end product.
They eventually tested Emiliano Martinez as Jack Harrison stung the palms of the stand-in Gunners goalkeeper, who was then helpless as Patrick Bamford's strike cannoned back off the crossbar moments later.
Arsenal – as they have been so often this season – were creating their own problems as a number of misplaced passes out of defence from the likes of David Luiz and fit-again Rob Holding saw them caught out by a Leeds side full of pressing players.
Arteta's surprise decision to deploy Sokratis Papastathopoulos at right-back was also looking questionable as the Greece international was easily beaten for pace by Ezgjan Alioski, who then angled a shot wide.
Despite all of Leeds' attacking play, they should have been behind on the half-hour mark but Luiz could only head Mesut Ozil's corner wide having been picked out free in the box.
At the other end, Martinez had to spring across his goal to keep out Alioski's close-range header after the winger once again got the better of Sokratis.
Yet another opportunity went begging for the away side as Robbie Gotts, another man making his Leeds debut, miscued a strike as the Arsenal fans began to grow restless.
That unrest continued as the half drew to a close and concluded with Sokratis appearing to gesture to a section of supporters to calm down.
Arsenal responded by starting the second-half with an urgency that was lacking for the opening 45 minutes, Nelson bursting into the box and then Alexandre Lacazette clipping the crossbar from a free-kick.
Lacazette, wearing the captain's armband in the absence of strike partner Pierre-Emerick Aubameyang, then drew a low save out of Meslier before Matteo Guendouzi's curling shot bent wide of the post.
Leeds were now the side playing on the counter-attack and Bamford tested Martinez as a frenetic opening to the second period continued.
Arsenal broke the deadlock from the very next attack as Pepe burst forward and fed Lacazette, who crossed in for Nelson to bundle home despite close attention from Meslier.
Lacazette was lucky to escape a red card after aiming a kick at Gaetano Berardi, with both referee Anthony Taylor and the video assistant referee happy that it was not a matter of violent conduct.
But there would be no late salvo from a Leeds side who have made something of a trademark from scoring late goals and they will return to their promotion push as Arsenal prepare for a trip to the Vitality Stadium in round four.