Although the visitors dominated much of the opening period, the Royals snatched a 17th-minute lead thanks to Alfa Semedo's audacious long-range strike.
Reading had been in good form, unbeaten in seven Sky Bet Championship matches until losing 3-1 at home to Brentford on Wednesday evening.
Evans, another ex-Royal, guided his header towards goal but, with home keeper Rafael well beaten, it thudded against the crossbar.
Reading gradually improved and went ahead nine minutes later, the goal tinged with a touch of fortune.
Millwall keeper Bartosz Bialkowski threw the ball to Williams but the midfielder did not appear to realise that it was coming to him.
Semedo nipped in and, from 35 yards out, the Benfica loanee beat the backtracking Bialkowski with a speculative yet successful first-time finish.
Reading maintained the pressure towards half-time but Millwall could have equalised from a Malone free-kick – Cooper found space at the far post only to nod weakly wide.
Ben Thompson also had a chance, from the edge of the home area, but volleyed straight at Rafael.
Neither side seemed willing to push too far forward at the start of the second half, the contest increasingly fought out in the middle of the park.
Millwall did attempt to up the tempo but could still not find the target, with Evans volleying over from a good opening.
Reading replied from a quick break but Olise, with time to pick his spot, shot tamely at Bialkowski.
Millwall pushed forward strongly thereafter and were rewarded in the 76th minute when Smith jabbed home from close range after Tom Bradshaw's scuffed shot.
Ovie Ejaria struck the crossbar as Reading tried to respond quickly but, five minutes from the end, Bennett dashed their hopes when popping up to nod home the winner.