The 23-year-old Pompey academy graduate registered almost 60 appearances for Kenny Jackett's side.
But he turned down the offer of a new contract at Fratton Park and has now secured a two-year deal with Killie after impressing Rugby Park boss Alex Dyer during a run-out as a trialist in last week's friendly win over Motherwell.
He becomes Dyer's fifth new recruit of the summer and will now compete for the left-back slot with Calum Waters, who is back in Ayrshire after spending last season on loan at St Mirren.
Haunstrup told the Killie website: "The move has been in the pipeline for a little while and I couldn't wait to get up here and I'm delighted it is all sorted.
"As a player, I'm energetic and like getting forward and I'm happy playing left-back or as a wing-back. The manager, coaching staff and players all seem great and I can't wait to get started."