The Bulls showed little of their most convincing form yet romped into a two-goal lead through Craig Stanley and Adam Stansfield before Adam Garner pulled a goal back for the visitors just before the half-hour.
After a scrappy opening, Hereford took the lead with their first serious attack of the game on 12 minutes.
Jamie Pitman delivered a free-kick into the Forest Green penalty area and the ball was half-cleared to Stanley who rifled in a powerful drive from the edge of the box.
The Bulls extended their lead eight minutes later when a Danny Williams pass was deflected to put Stansfield in the clear.
The former Yeovil striker rounded a defender before cracking in a left-foot drive past Dean Williams.
After hardly being in the game for the first 29 minutes, the visitors surprised everyone by pulling a goal back when Garner volleyed in Matt Gadsby's cross.
Rovers could have been on terms a minute later when a Garner volley flashed past a post with Craig Mawson beaten.
After the break, Hereford continued to have the better of play and David Brown was only inches away from extending the lead when he stretched to reach a Stansfield cross.
The Bulls, however, had a lucky let-off when Lee Mills, a substitute for Brown, was well-placed to hack a header from Garner off the line.
But Hereford were unfortunate not to add a third goal in stoppage time when Danny Carey-Bertram's shot, with the whole goal to aim at, hit the sole defender and was cleared to safety.