Fraser's first-half effort has seen Bournemouth rise into 12th position in the Premier League table, and left relegation-threatened Swansea in 17th - just one point clear of 18th-placed Southampton.
The home side had a half-chance inside the first two minutes when Callum Wilson broke into the Swansea box, but Fernandez was across to make a fine challenge.
Bournemouth had the ball in the back of the Swansea net in the ninth minute when Pugh struck home from close range, but the offside flag cut the celebrations short.
Swansea's first chance of the match arrived in the 12th minute when the ball fell for Mike van der Hoorn inside the Bournemouth box, but the centre-back's effort was saved by Cherries goalkeeper Asmir Begovic.
Wilson then turned a brilliant Fraser cross wide of the post down the other end as opportunities continued to arrive.
Martin Olsson was next to try his luck for Swansea as he headed over just before the 30-minute mark, but it was Bournemouth that made the breakthrough in the 37th minute.
Jordan Ayew had another opportunity moments into the second period, and his strike brought a smart save from Begovic.
Carroll struck over the home side's crossbar 20 minutes from time as Swansea looked for a route back into the match, but Bournemouth were managing the game, and the Cherries came close to a second when Nathan Ake headed towards goal in the 74th minute, only for Fabianski to make the save.
Swansea had a lot of the ball in the latter stages as they looked for a leveller, but it was Bournemouth that came closest to scoring the game's second goal, with Fabianski making a wonderful save to deny Wilson in the 85th minute.
The Welsh outfit will drop into the bottom three if Southampton take something from their clash with Everton at Goodison Park.
BOURNEMOUTH (3-4-3): Begovic; Francis, Cook, Ake; Fraser, Cook, Surman, Daniels; King (Mousset 92'), Wilson (Mings 94'), Pugh (Gosling 78')
SWANSEA (3-4-3): Fabianski; Van der Hoorn (King 72'), Mawson, Fernandez; Roberts, Ki (Narsingh 77'), Carroll, Olsson; Dyer (Abraham 59'), J. Ayew, A. Ayew