Trees quite easily grow up out of hedges, which of course are made largely of tree species. On the other hand trees cannot grow where land is regularly ploughed, mown or grazed.
Sometimes the hedge itself has been removed, leaving just the trees. Trees in the middle of fields are often left from when there were more fields divided up by more hedges.
You also see the same effect in woods (in fact I was surveying one like this in Essex yesterday). Very often the trees in the body of the wood are smaller than the ones round the edges. This is either because the main wood has been coppiced (cut down and allowed to regrow), or because it is a field which has become overgrown, and the "hedge" trees were already there beforehand.
One place you do get large trees in the middle of fields is where ancient parks are being farmed. There, some of the old parkland trees remain scattered about amongst the crop, and of course there never were any hedges.