The match looked to be heading for a 1-1 draw when Stuart rose to power home a header from Stuart Thurgood's final minute cross.
It must have been a huge relief to Stuart, who had earlier blundered to let Rovers back into the match.
Former QPR and Swansea forward Dennis Oli fired the visitors ahead after 18 minutes when he slotted home from a John Nutter centre.
However, Stuart gifted the hosts the opportunity to get back into the match when he handled in the area five minutes later.
Ex-Cambridge United stalwart Paul Wanless gratefully accepted the chance to level from the spot.
It appeared Wanless' effort would earn Rovers a third consecutive draw, until Stuart's header condemned them to their second defeat of the campaign.
Grays' spell at the top of the table may prove to be short lived as former leaders Exeter face Dagenham on Saturday, but Mark Stimson must be delighted with his side's unbeaten start to the season.