Aston Villa claimed just a second victory over Manchester United at Old Trafford in 38 years after Bruno Fernandes missed an injury-time penalty.
The game had looked set to end goalless until the 88th minute, when defender Kortney Hause flicked a near-post header beyond David De Gea.
Hause, in the side because Axel Tuanzebe was ineligible to face his parent club, went from hero to villain in the space of a couple of minutes when he handled Fernandes' cross but the playmaker sent his spot-kick high over the bar.
It was a second home defeat of the week for United and a first in the Premier League this season, while they also lost Harry Maguire and Luke Shaw to injury in front of watching England boss Gareth Southgate.
Ole Gunnar Solskjaer spoke in his programme notes about wanting to avoid another slow start after the disappointment of Carabao Cup defeat by West Ham in midweek.
He made 11 changes, reverting back to the team that beat the Hammers in the league last weekend, while Hause was the only change from Villa's victory over Everton.
United were soon on the attack and a good break down the left involving Shaw and Paul Pogba ended with Fernandes scooping a shot well over.
Cristiano Ronaldo had his first sight of goal in the 11th minute, sending a shot wide, but it was Villa who should have taken the lead five minutes later.
Matty Cash caught out Shaw and broke away down the right before sending a low ball across the six-yard box to Matt Targett, who poked over an open goal from close range.
An immediate response from United saw Pogba find Greenwood, but his shot was easily saved by Emiliano Martinez.
Maguire put De Gea in trouble with a misplaced back pass that the keeper could only direct to Ollie Watkins but the Villa man sent a shot straight at the Spaniard, who gratefully gathered the rebound off Danny Ings.
Villa's defence were frustrating United, with Pogba the latest to see a shot deflected wide in the 23rd minute.
Having earlier received treatment, Shaw was forced off 11 minutes before the break, with Diogo Dalot coming on, and the chances continued to arrive at both ends.
In the 39th minute, Douglas Luiz won a free-kick and then saw his goal-bound effort headed wide by Maguire, with Ezri Konsa heading over from the resulting corner.
Martinez produced his first serious save in the 43rd minute, pushing away a Maguire header from Fernandes' free-kick, while United should have taken the lead in injury time but Pogba guided his header from a corner just wide with Martinez helpless.
Maguire was forced off 20 minutes into the second half after having his left calf strapped, Victor Lindelof slotting into defence.
Villa were the better side after the break and De Gea dived to his right to keep out a dipping shot from Watkins, while Jacob Ramsey would have had a clear sight of goal from 12 yards but for an ill-timed slip.
United began to threaten again as the game headed into its final stages, and one of the bigger cheers of the day was for the introduction of fit-again Edinson Cavani as a late substitute.
But it was Villa who made the breakthrough before dramatically hanging on for a deserved three points.
