Hitler had in fact planned an invasion of Britain, but was aware that he first had to gain arial superiority and wipe out the RAF. At the same time ha also believed that Britain would eventually succumb to the Nazi war machine sooner or later.
During the Battle of Britain, had the germans waged an all out war it's quite possible that the RAF could have been defeated, but tactical mistakes made by the Luftwaffe's high comand in the use of the Messerschmitt fighter, and the under estimated importance of the British early warning radar system gave the RAF the small advantage they needed, and with which, they inflicted heavy losses to the German Luftwaffe.
This convinced the german leaders to give up the direct conflict with the RAF (bringing an end to the Battle of Britain) and instead, they turned thier attention to the night bombing of cities in the hope of flacking the British morale. This too proved to be a failure despite of the high loss of lives among the civilian population.
Towards the end of October of the same year, the Germans were also invoved on various other fronts and given the situation, they decided to leave the British Isles to one side in the conviction that "things could be taken care of " the following spring. This fortunately never happened as the RAF had the breating space to train more pilots which were desperately needed, and the Germans were involved on so many different battle fronts that never again were they able to put up a force strong enough to invade Britain.
.