A late David Wotherspoon strike drew Saints level on points with the seventh-placed Ayrshire club.
The result leaves Dyer without a point since taking over the reins last month from Italian boss Angelo Alessio.
A controversial Chris Burke spot-kick had hauled Killie back into contention after midfielder Alistair McCann's seventh-minute opener.
But the visitors lacked a cutting edge and spent most of the second half on the back foot.
"Did we deserve something? I'm not 100 per cent sure, to be honest," said Dyer, who admitted the travelling fans had been let down.
"We are lacking confidence at the moment as well as quality so it was probably a fair result."
Dyer, who played a supporting role to Steve Clarke last term as Kilmarnock secured European qualification, warned there are no quick fixes.
"If you keep losing games then there is something fundamentally wrong – that is a rot.
"We have to look down these days because we are closer to the bottom than the top six where we want to be.
"We need to change what we're doing. We have changed the manager already and unless we are going to change it again then we have to dig in and go again.
"I have no concerns. I know what I have in the changing room and what I need to do.
"If we do our best then it will be OK – there is no need to panic."
St Johnstone manager Tommy Wright might have lost top-scorer Matty Kennedy to Aberdeen and griped about the lack of signings sanctioned by the club hierarchy in the winter window in recent days, but the Northern Irishman guided his injury-hit squad to a vital win.
He was furious with referee Alan Muir's decision to award Killie a penalty when Callum Booth's connection with Burke appeared to start outside the box.
And a hamstring injury sustained by Drey Wright stretched his squad even further in advance of Wednesday's visit from league leaders Celtic.
But Wotherspoon's late winner hauled Saints nine points clear of bottom club Hearts.
With Murray Davidson ruled out by a broken arm and on-loan Anthony Ralston unable to face his parent club, the Saints manager is hoping to secure Jamie McCart from Inverness after the defender penned a pre-contract.
Wright admitted he may have "ruffled a few feathers" after criticising his club's signing strategy.
He said: "What I've always tried to do is look after my club and speak passionately about my club.
"At times we could be more proactive than reactive. We're waiting until the last seven days of the window.
"We do need bodies in."