The 34-year-old left the Etihad Stadium last May after more than a decade to take up a player-manager role at Anderlecht in his native Belgium.
Kompany's first season with Anderlecht has not gone entirely to plan, though, and The Sun claims that he held talks with Guardiola over a possible return to City.
Guardiola is said to have lined up the experienced defender to succeed Mikel Arteta, who stepped down as City's assistant boss in December to take up the top job at Arsenal.
However, the report indicates that Kompany turned down the opportunity as he still wants to balance coaching with playing, which he allegedly felt would not be possible with City.
Kompany made 360 appearances for the Citizens in all competitions across 11 seasons.