The 41-year-old spent two-and-a-half years in Major League Soccer with NYCFC, a sister club of Manchester City, as part of his coaching development.
Man City announced on their official website on Sunday evening that Vieira will step down from the position with immediate effect, although the web page was removed soon after.
Vieira's statement read: "I would like to thank City Football Group and New York City FC for the opportunity to have coached this tremendous football club. Leaving New York is an incredibly difficult decision for me and my family and one that has not been taken lightly.
"From the beginning, City Football Group and its leadership, particularly our chairman Khaldoon Al Mubarak, Brian Marwood, Ferran Soriano and Marty Edelman, have believed in me and have helped guide me through the last eight years of my career - starting with my time as a player and ultimately culminating with my first opportunity to manage a first team."
Nice, who finished eighth in Ligue 1 last season, released a statement of their own on Monday afternoon to confirm Vieira's arrival.