Yes, Vannon is quite right.
The government is not "busy closing the door on eastern Europeans coming here to work,". What it wants to do (spurred on by the referendum result) is to remove the country from its obligation to allow anybody from other EU countries to live and (possibly) work here. It wants the UK government to be in charge of immigration from the EU as it is in charge of immigration from elsewhere. It wants to be able to say to people like the Romanian gypsies, currently living in the pedestrian underpasses at London’s Marble Arch, that since they have nothing useful to contribute to the UK’s economy and wellbeing, nor anywhere to live and that they are in fact a nuisance, they are not welcome here and will not be admitted. It also wants to be able to say to people like Charlie Mullins, boss of Pimlico Plumbers, that if he cannot find suitably qualified people for his workforce in the UK then he can apply for some work permits for people to come from Poland (or wherever else he feels he can source his employees from).
Few sensible people who voted for Brexit believe that our departure will see an end to all migration from the EU. Indeed few sensible people would want to see an end to it. Charlie Mullins will have to make a little bit more effort to source his workers and if he really cannot find any already in the UK he will still be permitted to import them. But the net result will benefit the UK as a whole. What most sensible people who voted to Leave want is an end to having no control over the entry to the UK of people like this lot:
https://refugeeresettlementwatch.wordpress.com/2014/10/16/romanians-living-in-london-subway-demand-homes/
(I know the article is a couple of years old but they are still there, believe me. I saw them at the end of November)
In fact, sensible people who voted to Remain would probably like the same. But as Mr Cameron found out when negotiating his much-heralded “reforms” to the UK’s relationship with the EU, that is not a possibility whilst we are members.