The Hoops had to come from behind to beat St Mirren in the Scottish Premiership in midweek and again on Saturday they lost the first goal – to a Jason Holt penalty.
Hoops midfielder McGregor levelled with a drive and further goals from Ryan Christie and Albian Ajeti had Neil Lennon's side comfortably ahead and in control of the match until Julien Serrano pulled a goal back for Livi in the 78th minute.
That ensured a nervy finish in which Livingston's debutant substitute Carlo Pignatiello missed a late chance for the visitors.
If Celtic triumph against Riga in Latvia they will play FK Sarajevo or Buducnost Podgorica away in the play-off and McGregor is looking for a clean sheet to help them progress.
He told CelticTV: "We probably made it more difficult than what it should have been.
"At the start of the second half we look good, we look comfortable, get the third goal and then you probably fancy us to see it out 3-1 or 4-1 but I think that's something that we have to really work on, in terms of conceding goals.
"We're conceding too many goals as a team, shots from the edge of the box and things like that which is not really us
"We have to work on that, we'll need to look at it and make sure going forward that we don't concede as many goals.
"We know what we need to do on Thursday.
"We look at this (Livingston) game, we have to learn from it quickly in terms of conceding goals.
"We go over there and put everything into the game and we are looking to get into that play-off spot so everyone's looking forward to it."
The Scotland international was pleased to get his first goal of the season from a deflected drive from the edge of the box.
He said: "I thought the boys reacted really well actually to the penalty.
"We got going and the ball fell to me and I just took a touch and hit it and sometimes they get a little nick when they go in which is fine, I will happily take that one."