The striker pounced after just eight minutes of play at Sandy Lane, the home of Worksop Town.
After three defeats and one draw in the opening fortnight of the season, manager Ryan Austin made a change to the system the team played. He went for a 4-4-3 system with Anthony Malbon, Gyamfi and Ruben Wiggins-Thomas starting as a front three.
The first real chance of the game was Gyamfi's goal on eight minutes. Wiggins-Thomas was able to beat his man before putting the ball into the area. The unmarked Gymafi pounced to slot the ball home.
Kyle Blake and James Bulter had chances to increase the Grove's lead. Blake put his header straight at David Reay. Meanwhile, Butler put his header just wide of the goal from a Charlie Scott corner.
Andrew Gascoigne had an excellent chance on 38 minutes to equalise for the hosts. The ball was cleared out to him and, he thumped his effort against the post which left Kieran Harrison rooted to the spot.
The second half saw Worksop get on the front foot with Craig Mitchell having two chances which he put over the bar. Daniel Patterson also put a chance over the bar from the corner.
Those chances were the only ones that Worksop had as the Grove defended to a man keeping them out despite the hosts having most of the possession.
They needed to defend well as many supporters will remember the likes of Runcorn and Prescot away last season where the Grove threw away a 1-0 lead in the final few minutes.
Malbon should have had a penalty on 86 minutes when he went down in the penalty area. However, the referee felt that he had dived and was cautioned.
Kidsgrove had two great chances to win the game in the dying moments. First James Jennings hit a chance wide of the post after he broke clear. While Max Murphy put an effort just over the bar in the second minute of injury time.
It didn't matter as the goal from Gyamfi was enough to give Kidsgrove their first league win of the season and it will set them up nicely for the FA Cup replay at Newcastle Town on Tuesday night.
Kidsgrove Athletic: Harrison, Thompson, Morris, Hall, Duffy, Butler, Scott (Murphy, 60), Blake, Wiggins-Thomas (Jennings, 69), Malbon and Adu Gyamfi (Bergin, 84).
Unused subs: Rhodes and O'Neill.
Worksop Town: D.Reay, Rhodes, Smith, Wolley, M.Reay, Wankiewicz (Patterson, 55), Pemberton, Gascoigne, Mettam (Palmer, 55), Mitchell and Karkach.
Unused subs: Brunt, Jordan and Atherton.
Attendance: 332