Without their most influential player they followed successive defeats to end the wider run of 13 league defeats that had come when Zaha was unavailable.
During those 13 fixtures — their last league success without him came at struggling Sunderland in September 2016 when Alan Pardew remained manager — they had scored only three goals and conceded 28.
The lack of a consistent goalscorer remained a concern at Selhurst Park but their captain Milivojevic produced a fine first-half finish to give them the lead they retained to move up to 15th and clear of the Premier League's bottom three.
Leicester had earlier perhaps been unfortunate not to go in front midway through the first half when Jamie Vardy found the back of the net but was judged to have fouled Palace's Spanish goalkeeper Vicente Guaita, handed his Premier League debut after back spasms ruled out Wayne Hennessey.
Instead of clearing Milivojevic's pass-back, Guaita took a heavy touch and changed direction, tempting Vardy to pursue the loose ball he won after a shoulder barge that knocked over the goalkeeper, who was awarded a free-kick just as Vardy finished.
Beyond a speculative effort from the largely listless Jordan Ayew and the diving Guaita saving a header from Vardy, there had been little in the way of a threat on goal from either team, and it was a similar lack of options that essentially led to Palace taking the lead in the 39th minute.
With Aaron Wan-Bissaka and Andros Townsend exchanging passes near the Leicester penalty area as they pursued a target, Townsend found Milivojevic, who in turn unexpectedly looked up and brilliantly bent from 25 yards just inside the far right post and beyond Kasper Schmeichel's fingertips.
It has been a lack of goals, in the absence of the injured Christian Benteke, that has undermined Palace this season but taking the lead again gave them belief, even if a second goal rarely looked likely.
After they lost Cheikhou Kouyate to injury Demarai Gray and Vardy combined to create a shooting chance for Leicester's Rachid Ghezzal, who struck wide, and it was Vardy who then so nearly scored.
On the counter Ghezzal linked with fellow substitute Kelechi Iheanacho, who played Vardy clean through on Guaita, but when the striker struck towards the bottom left corner Guaita reacted to dive and tip the ball onto the post before collecting it as the rebound came back into his hands.
A final, late chance fell kindly to Wilfred Ndidi on the edge of the area, inviting the midfielder to strike, from where he connected well and with power but shot beyond the crossbar, leaving them narrowly outside of the top 10.