The Blues struggled for large parts of Thursday's game at the Sinobo Stadium, but Alonso's header four minutes from time has provided the Blues with a potentially-decisive advantage ahead of next week's second leg at Stamford Bridge.
Slavia Prague netted six times over two legs against Sevilla in the previous round, and the Czech outfit went close to their first of this tie after less than two minutes as Simon Deli sent a header wide of the far post from eight yards out.
Maurizio Sarri had made seven changes to his starting lineup and the alterations perhaps contributed to a disjointed opening from the visitors, although Willian was offering a threat down the left side of the pitch.
Willian went closest for the Blues in the 25th minute, with the Brazilian cutting inside onto his right foot on the edge of the area before striking the crossbar from 20 yards out.
Slavia went back on the front foot during the closing stages of the opening period, but Chelsea were doing an effective job in keeping their hosts away from goal.
Chelsea returned for the restart with the intent of offering more of a threat than they did before the break, but Olivier Giroud scuffed a shot wide of the post from eight yards after being found by Willian.
Sarri was visibly unhappy with how his players were performing, and the Italian made the decision to introduce Eden Hazard from the substitutes' bench before the hour mark.
The alteration acted as the catalyst for an improved spell from the visitors, with Antonio Rudiger seeing a volley kept out by Ondrej Kolar and Willian dragging a low effort well wide of the far post.
Rudiger also headed off target from a corner, and there was every suggestion that Chelsea were ready to push forward and dominate the closing quarter of the contest.
However, Slavia responded positively to being put under pressure, and Arrizabalaga was required to dive to his right in order to keep Ibrahim Traore's strike from finding the top corner.
Jan Boril also saw a volley from inside the area kept out by the goalkeeper, but Chelsea landed the telling blow through Alonso with just four minutes left.
Willian's outswinging cross from the right flank picked out the run of the Spaniard, and the left-back did the rest by powering a header into the bottom corner of Kolar's net.
Slavia pushed forward in search of what would have been a deserved equaliser, but Chelsea comfortably held on to put themselves in a strong position to reach the last four.
SLAVIA PRAGUE (4-2-3-1): Kolar, Coufal, Ngadeu-Ngadjui, Deli, Boril; Traore, Kral; Masopust (Husbauer 74'), Sevcik, Stoch (Zmrhal 64'); Olayinka (Van Buren 82')
CHELSEA (4-3-3): Arrizabalaga; Azpilicueta, Rudiger, Christensen, Alonso; Barkley (Loftus-Cheek 75'), Jorginho, Kovacic (Kante 68'); Willian, Giroud, Pedro (Hazard 59')