James Morrison handed the visitors the lead just before the half-hour mark, but Gareth Bale equalised from the penalty spot in the 81st minute after it was deemed he had been fouled in the box by Shaun Maloney, and then rifled in a delightful winner with a minute of normal time remaining.
The result brought the Dragons' five-match losing streak since Coleman had been appointed as manager to an end as they bounced back from a 2-0 defeat to Belgium and humiliating 6-1 reverse in Serbia in their opening two Group A fixtures to kickstart their campaign. Reflecting on the performance, Coleman said: "This will give them confidence."
He added: "You can start feeling sorry for yourself, thinking 'this is never going to happen' or 'we are unlucky again' or 'it is not our time' or whatever.
"It takes a lot of courage to keep going at 1-0 down. It was the nature of how we won the game - in the last 10 minutes after being 1-0 down for so long and getting into some really good situations and not making the most of them.
"We just kept going. We never panicked or kept knocking it long. At the end of the day, it is all about the players - they had to produce. The pressure had come on all of us, and the players felt it going into this game. But they couldn't have reacted any better - I thought they were magnificent and deserved it."
Wales' win moves them off the bottom of the group and up to fourth, behind leaders Belgium and second-placed Croatia, who both have seven points from three games, and Serbia, third on four points. While that will have eased the heat on Coleman, it is only being turned up on his opposite number Craig Levein, whose team are fifth with two points following their home draws with Serbia and Macedonia - the new rock-bottom side - last month.
Levein's main complaint at the end of the match was over the decision to disallow a headed effort by Steven Fletcher in the second half which would have made it 2-0. Fletcher, back in the side for the first time in over two years, nodded in a cross from Charlie Adam which was adjudged to have gone out of play.
Replays suggested otherwise, and Levein said: "With 10 minutes to go I just could not see us losing the match. Getting the second goal was always going to be crucial, though, because a 1-0 lead can be fragile, and so it proved tonight.
"That second goal was so important for us, and I believe the officials made a serious error with the chalked off goal that Steven Fletcher scored. That is very hard to take."
Source: PA
Source: PA