Two lapses in the Forest Green defence and the breakaway goal by Danny Williams gave Hereford United a 3-1 victory, with the home side being reduced to 10 men with Steve Jenkins being sent off 13 minutes from time for his second bookable offence.
Rovers started well with Neil Grayson shooting wide after 13 minutes when Alex Sykes may have been the better option with the winger wide open on the right-hand side.
Forest Green did take the lead three minutes later when Steve Perrin's long goal-kick was flicked on by Sykes into the penalty area and Grayson hammered the ball from a narrow angle into the bottom left-hand corner of the net.
Grayson almost grabbed the second after 25 minutes when Matthew Coupe's indirect free-kick was headed down by Jamie Impey and Grayson with his back to the goal sent an acrobatic shot over the bar.
A long Hereford ball against the run of play a minute later caught the Forest Green defence napping and Paul Parry managed to edge past the last defender before slotting the ball out of Perrin's reach into the back of the net.
Parry's goal lifted Hereford who had looked subdued up until that point and they looked capable of taking the lead with Rob Sawyers' effort from the edge of the penalty area going just over the crossbar 13 minutes into the second half after a poor Forest Green clearance.
But a minute later they deservedly went ahead when a Michael Rose corner found captain Ian Wright, whose header took a deflection into the back of the net.
Rovers had a golden chance to equalise 15 minutes from time with captain Martin Foster beating the Hereford offside trap and rounding Hereford keeper Matt Baker only to fire the ball over the empty net.
Soon after Jenkins received his second yellow card after earlier being booked for ungentlemanly conduct, when he was a second too late with a tackle on Steve Guinan and the referee had no alternative but to give him his marching orders.
Hereford sealed the win deep in injury time with all of the Rovers players pushed up to find an equaliser. Parry broke free only to see his shot well saved by Perrin, but Danny Williams had the simple task of tapping the ball into the back of the empty net