The Gunners were heading for a first home defeat to lower-league opposition in this competition since November 1983, until Arsene Wenger turned to his young wildcard with six minutes of normal time to play.
Arsenal, a goal down to a Josh Murphy strike, relied on the 18-year-old forward to first force extra time by netting from close range with his first touch, before heading an extra-time winner for the two-time winners.
Neither side could muster a serious shooting chance in a quiet start to the match, with the first attempt arriving via the boot of Alex Iwobi 20 minutes in for an Arsenal side showing 11 changes from last time out.
The Nigeria international fired well over the bar, while up the other end Mario Vrancic dragged wide when he had options better positioned either side of him.
Heading into the match on the back of a nine-game unbeaten run, and with five successive away victories to their name in all competitions, Norwich were managing to match their opponents all over the field when the breakthrough moment arrived.
James Maddison, the derby-day hero against Ipswich Town at the weekend, slotted a perfectly-weighted through-ball for Murphy to latch on to, with the winger dinking the ball over the outrushing Matt Macey to make it four goals in four EFL Cup games this season.
City could easily have had two soon after, only for Macey to this time produce a top save to push aside a curled effort from Nelson Oliveira - one of four fresh pairs of legs to be brought into the side on the back of the win at Portman Road.
Theo Walcott was thwarted by Manchester City academy graduate Gunn with a couple of weak shots at the end of the half, as Arsenal went into the break a goal behind.
With six of his seven subs being inexperienced teenagers, Wenger decided to stick with what he had and Arsenal did begin to see more of the ball in the final third.
Norwich had chances of their own to extend their advantage, though, including an opportunity for goalscorer Murphy when ghosting in behind at the back post and seeing his shot from an Ivo Pinto cross kept out.
At that point the Gunners could perhaps consider themselves lucky to still have 11 men on the pitch, as Mohamed Elneny dragged down Oliveira when the last man but got away with just a yellow card.
Oliveira, a fitness doubt before kickoff, was proving to be a real nuisance as he peeled off his man and blazed over when picked out unmarked in the box by Pinto 70 minutes in.
As time ticked down it was the Norfolk outfit creating the killer chances, one of which Vrancic again failed to take as he cut inside and blasted into the side-netting, with two teammates screaming for the ball to be squared.
Walcott headed onto the roof of the net as the final 10 minutes arrived, but Arsenal did draw level with five minutes to play through Nketiah, who stabbed home a flicked corner 15 seconds after being introduced.
Wilshere could have won the match in normal time, being brilliantly denied by Gunn from five yards out, while substitute Chuba Akpom also came close.
Another opening came and went for Walcott early on in extra time, but Nketiah had less trouble with his finishing as he nodded the ball past Gunn from a left-sided corner.
Nketiah nearly bagged the unlikeliest of hat-tricks after being played through in the second half of extra time, with Gunn again doing his first-team chances no harm by producing a big stop.
There was still time for one more twist, however, as James Husband was illegally brought down inside the box by Mathieu Debuchy late on - referee Andy Madley adjudging that not enough contact was made, meaning a place in the last eight for Arsenal. body check tags ::