The win saw Grove move up to fourth place, but they now go into the Christmas period without top scorer Walker, who is banned for three games.
Manager Pete Ward said: "Their player has gone down like a sack of spuds, and the referee has shown Dave a straight red.
"We'll obviously miss him, but we've got Liam Shotton to come in, and he's always a threat."
Kidsgrove found themselves a goal down on 10 minutes after Goole's Simon Spencer struck, but they equalised three minutes later through Walker.
Hot-shot Walker put Ward's men 2-1 up on 29 minutes before Adam Beasley made it 3-1 before half-time.
Goole got themselves back in the game straight after the restart with a Simon Harrison effort.
It was soon after that Walker was ordered off, and Goole made the most of their one-man advantage to equalise when Spencer bagged his second goal of the afternoon.
It looked as though Kidsgrove would face an uphill task to get anything from the game at that point, but goals from Dan Schwarz and Karl Charlton saw Ward's men take a deserved three points.
"When they equalised to make it 3-3, we were down to 10 men and it looked as though they would go on to win," Ward said.
"But we've dug in and really deserved our victory in the end."