After making two mistakes which directly contributed to the Baggies establishing a 3-0 lead at half time, Lampard opted to replace the Spaniard.
The left-back was expected to join the rest of the Chelsea squad in the stands for the second 45 minutes at The Hawthorns.
However, The Athletic claims that Alonso attempted to get on the team coach soon after his withdrawal, a move which angered Lampard.
The report quotes a source as saying that the Blues boss was 'the angriest that they have ever seen him' during his time as manager.
Lampard will likely part ways with either Alonso or Emerson Palmieri before the end of the summer transfer window.