Carson saved three penalties out of three to secure Motherwell's first shoot-out win since 1999 following a 2-2 draw in the second qualifying round.
First-half goals from strikers Callum Lang and Tony Watt had put Motherwell in command at The Showgrounds.
But Ben Doherty netted from the spot inside four minutes of the second half and he repeated the feat in the last minute after Bevis Mugabi had been sent off for a last-man foul.
However, Carson proved unbeatable in the shoot-out as Motherwell sealed a tie against Hapoel Beer-Sheva with the one-off game due to be held in Israel, which is currently under lockdown, next Thursday.
Coleraine's stunning shoot-out win over Maribor had propelled Motherwell among the seeded teams before the pair drew each other.
Stephen Robinson's Motherwell beat Glentoran 5-1 in the previous round and they started strongly, with Mark O'Hara and Lang having efforts blocked amid some desperate defending.
However, Coleraine missed a good chance just before the opener. James McLaughlin, looking for his third goal in three games in the tournament, headed wide following a free-kick after getting ahead of Ricki Lamie.
Motherwell went straight upfield and netted a 16th-minute opener. Watt beat a man and crossed and the ball was headed back in by Stephen O'Donnell for Lang to net from close range at the second attempt.
Motherwell defender Declan Gallagher saw an effort cleared off the line before Watt doubled Motherwell's lead in the 37th minute. The former Celtic striker fired a first-time effort into the bottom corner from 16 yards after Lang's cross fell for him off Liam Polworth's touch.
McLaughlin missed another headed chance in first-half stoppage-time but Oran Kearney's side got a goal back four minutes after the restart when O'Hara was penalised while tracking Doherty at a set-piece.
The Northern Ireland Under-21 international picked himself up and dispatched the penalty into the bottom corner.
Motherwell were suddenly under pressure and they had a let-off when Josh Carson was flagged offside after knocking the ball across goal for Gareth McConaghie to net.
Motherwell boss Stephen Robinson switched from playing three centre-backs to a 4-3-3 with the introduction of winger Sherwin Seedorf.
The tweak allowed Motherwell greater control but there was a warning when Stephen Lowry missed a great chance after losing marker Watt at a free-kick, and there was a further sting in the tail as Eoin Bradley got away from Mugabi from a flick.
The Coleraine substitute slowed down and drew contact with the defender before going down and Finnish referee Antti Munukka instantly pointed to the spot and showed a red card. Doherty sent his penalty into the other corner to take the game into extra time.
Motherwell brought on Christopher Long and he came close after bursting through the middle but Trevor Carson kept Motherwell level when he touched Bradley's header on to the bar.
He then saved from Ian Parkhill, Lyndon Kane and McConaghie as Well edged through with a 3-0 shoot-out win.