A third-round tie low on quality was settled by Josh Brownhill's 72nd-minute strike, the midfielder dispatching his fourth goal of the season after twisting and turning in a congested penalty area.
Until then, after a largely forgettable first half, Huddersfield had looked the more likely winners with half-time substitutes Alex Pritchard and Steve Mounie adding pace and purpose to their attacks.
But Huddersfield are bottom of the Premier League for a reason, they have scored only 13 times in 21 top-flight games and have little, if any, confidence in front of goal.
Against Bristol City, opponents unbeaten in eight Sky Bet Championship games, and increasingly finding form as the season unfolds, it was to prove their undoing.
The Robins were a constant set-piece threat through Adam Webster and, although their goal attempts were sporadic, Callum O'Dowda's first-half header did strike the woodwork.
Huddersfield handed a debut to Jason Puncheon following his loan move from Crystal Palace on Friday, and the veteran slotted in alongside Terriers skipper Jonathan Hogg in midfield.
Both sides struggled to find any fluency with much of the first half played almost at walking pace at times.
The tedium was only lifted by a muscular battle between City striker Famara Diedhiou and Huddersfield defender Mathias Jorgensen, which referee Peter Bankes had to keep a close check on.
Diedhiou rushed an opportunity when well-placed and Juninho Bacuna was similarly wasteful at the other end.
Chris Lowe's advances from left-back offered Huddersfield hope, but it was the hosts who had the first real chance of note after 36 minutes.
O'Dowda directed Jack Hunt's cross onto the bar and Diedhiou sent the rebound over under pressure from Jorgensen and Jon Gorenc Stankovic.
Jamie Paterson's fierce 25-yard drive then just cleared the bar before Huddersfield responded with their first incisive attack of the half.
Hogg slipped in Erik Durm and his cross was begging for a decisive touch, but the stretching Laurent Depoitre skewed it wide from four yards.
Depoitre and Bacuna were replaced at half-time as Terriers boss David Wagner, no doubt keen to avoid a replay with a relegation battle to face, sent on Pritchard and Mounie.
Pritchard appealed for a penalty within moments of his arrival after tumbling in the box, and there was suddenly more purpose about Huddersfield's play.
Lowe and Hogg fired wide and City were grateful for Bailey Wright's intervention after Mounie had lifted the ball over goalkeeper Niki Maenpaa and seemed certain to break the deadlock.
Pritchard wasted another promising opportunity, and it proved fateful as Huddersfield failed to clear a corner and Brownhill fired home past Ben Hamer at his near post.
Brownhill almost added a second when denied by a diving Hamer save, but City held on to take their place in round four.