Welsh aspirations of securing a place at the 2014 World Cup were effectively ended by Tuesday's 2-1 defeat to Croatia in Swansea, a result which leaves manager Coleman's side 10 points off the pace in Group A with only 12 up for grabs.
Skipper Williams said: "We definitely want him to stay, obviously I've been saying for a long time that he is doing a great job."
Wales have paid the price for a poor start to the campaign, having lost three of their first four games and, as with their Euro 2012 campaign, an improvement in form has come too late.
Coleman, whose current contract runs up to the 2014 finals in Brazil, recently admitted it had taken the 6-1 mauling in Serbia last September for him to come to terms with the task of succeeding Gary Speed following the death of his close friend.
Coleman had been loath to put his own stamp on the Wales set-up after vowing to continue Speed's legacy, but decided he had to start doing things his way to make the most of his time in the job.
Since then results and performances have improved, and the first-half performance against Croatia was the best of his reign to date.
Skipper Williams believes the recent upward trend means the Football Association of Wales should hand Coleman an extension to his contract, and says Wales are now producing the sort of form they showed towards the end of Speed's time in charge.
He added: "He came in and took the job in unbelievably tough circumstances, but I think you can see it's getting back to where it was under Gary Speed.
"The last two camps with him have been brilliant, I think that's really pleasing and it would be stupid to change the manager, no one would say they want to see a change."