Matt Smith's towering header had put the visitors in front in the 32nd minute during a first half in which both sides had limited threat.
Forest looked the more likely to win it amid a dramatic finale but the draw was a fair result, with striker Smith also having hit the bar for the visitors in the second half.
The home side made a bright start, with Lewis Grabban and Joe Lolley both only denied by last-gasp challenges in the box, as Cooper took charge for the first time following the departure of Chris Hughton.
When Djed Spence delivered a cross, Grabban managed to get on to the end of it but was denied by a strong save from Bialkowski.
The Millwall keeper and the Forest striker both needed lengthy treatment when they challenged for the ball after a Lolley long-range shot had been sent looping high into the air by a deflection.
Reds skipper Grabban, who was possibly feeling the impact of that earlier challenge, had to be replaced in the 27th minute, with Lyle Taylor coming on.
It was Millwall forward Smith who gave them the lead five minutes later in simple fashion. When Sheyi Ojo delivered a cross from the left, Smith rose powerfully to guide a header inside the post, with keeper Brice Samba a helpless onlooker.
A quick break then saw Brennan Johnson carry the ball to the edge of the Millwall box but, when he might have had a go, he unselfishly opted to pass to Taylor and the opportunity went begging.
Forest began the second half with Spence sprinting into space on the right and Johnson completely unmarked for a tap-in at the far post, but Spence could not pick out the pass.
The home side did draw level soon after though, in slightly fortunate circumstances, after Scott McKenna fed Lowe in space wide on the left. The wingback looped a cross towards the far post and it deceived Bialkowski completely before finding the top corner off the inside of the upright.
Forest then had a lucky escape when a poor ball out from keeper Samba led to Smith hammering a shot against the bar.
And there was almost a controversial moment when referee Jeremy Simpson did not stop play, despite getting in the way of a Forest pass on the edge of the box, with Millwall launching a quick attack which culminated in Samba saving a header from Smith.
Taylor had a shot blocked and Worrall forced a simple save from the rebound as the clock ticked down, with the home side looking the more likely to snatch victory.
Johnson also fired a superb ball right across goal, which somehow went unconverted, before Forest substitute Philip Zinckernagel put his effort narrowly wide as Millwall hung on for the point.