Joe O'Neill came close to giving the Minstermen an early lead, but after 15 minutes the visitors did hit the front.
Darren Dunning's corner from the left-hand side was turned home from close range past keeper Ryan Clarke by centre-half James Dudgeon.
Lewis Haldane had Rovers' best first-half chance to level, but Chris Porter was in the right place to gather his header.
After the restart Rovers struggled to find a way back into the game, before the introduction of sub Bruno Teixeira sparked them into life, with Porter needing to tip over a Luke Graham header.
However, within a couple of minutes City added a second as O'Neill set up Mark Convery to place a shot beyond the stranded Clarke.
With time running out, Teixeira gave Rovers hope as he converted Alex Meechan's cross and Jon Richardson and Zema Abbey both went close to snatching a point for the home side.