Grove welcomed the return of not only Matt Haddrell, signed from Congleton Town, but also Ricky Bridge, who responded with a brace.
Manager Peter Ward had loaned Bridge to Biddulph Vics to get some match practice on the understanding he was on instant recall, and he chipped in with his first goals of the season.
Ward said: "Ricky had got a couple of 90 minutes under his belt and his work rate was good.
"He always looks weary, but he got his reward and I was pleased for him."
Kidsgrove fell behind to a 28th-minute penalty that cost goalkeeper Steve Intihar a booking, but were soon level after Nick Ward set up Dave Walker from a set-piece.
Belper were threatening on the break and regained the lead on the stroke of half-time, after Dave Harper had gone to ground.
Walker levelled matters on the hour and Bridge weighed in with two late goals to seal an impressive victory.
Ward said: "It was an even game in the first half, although we were slightly on top.
"We bossed the game in the second and finished strongly.
"Haddrell was always a threat, but needs to adjust to the pace of the game at this level."
Kidsgrove host Colwyn Bay, second in the Premier Division, in the League Cup tomorrow (7.45pm).