Friday's 2-1 win over Scotland moved Wales up to third place in Group A at the halfway stage of the qualification campaign, although they trail pacesetters Croatia and Belgium by seven points. A play-off place may yet be achievable for Coleman and company, but Williams believes that possibility will evaporate if they cannot claim all three points at the Liberty Stadium.
He said: "We want to win the game first and foremost and see where we are after that. But we are seven points off them so if we want to do anything in the group we need to win this game, it's that important. We think we can win it, obviously we are trying to catch them and we want the points."
He added: "They are doing well in the group, they are battling it out with Belgium at the top and they will want the points as much as we do.
"But we will have to see how it goes and see where we are after tomorrow night."
Wales will have Gareth Bale available after the Tottenham star trained on Sunday and Monday, and he appears to be over the worst of the illness which saw him withdrawn at half-time in Glasgow, although Aaron Ramsey will miss out after his red card.
Wales made light of Bale's absence as they came from behind against Scotland, but Williams welcomes the 23-year-old's availability.
"He seems fine, he trained today and yesterday and he has been working closely with the doctors, getting his energy levels up and the right food down him," he said.
"He seems fine and I am sure he will be ready for the game.
"You always want him in the team, you could see he was struggling with illness in Scotland and he will hopefully be back to full fitness for Tuesday."