Manager Peter Ward blamed individual errors for Grove's slip.
He said: "We did not play badly. Conditions were horrendous for football because you could not get the ball down and play.
"Quorn were the better side for the first 20 minutes, but we got back into the game and scored a good goal when Matt Haddrell set up Karl Charlton (36 mins) from a set-piece.
"Tim Sanders then picked out Nick Ward with a free-kick to give us the lead just before half-time, and when Dave Walker added the third (55) the game should have been over.
"Charlton hit a post at 3-1, but with six minutes left they scored from a direct free-kick when we should have cleared, and in the last minute Sam Hall gave away a penalty.
"We are down to eighth and need to put a run of wins together to get back up there. But I think we are quite capable of doing so."
Kidsgrove are at home to fourth-placed Rushall Olympic tomorrow.