I think you are over-analysing things Scarlett. The tactile thing does't seem overdone if the hug was a light one rather than a big squeeze. The double-handed handshake can be a sign of sincerity or confirmation of friendship.
I've used both at times with men and women, just as a sign of friendship, thanks, goodbye etc., when it seemed appropriate.
Forget the contrast between the 2 meetings - he seems as confused as you as to where it's all going.
Are you the making a problem from nothing, Scarlett, with your hook-up on body language and eye contact? Were you overdoing the attempt at eye-contact to the extent that he felt stared at? Constant unrelenting eye contact seems abnormal to me - try looking around and at your meal etc.
If you need to meet regularly then go ahead but just leave out the analysis and see what happens like two adults. If he oversteps the mark set by you let him know at once and work out what happens then.
No-one has asked so hear goes: do you fancy him?
Finally my major criterion whatever the relationship is always:
Do they smile with their eyes? If not don't trust them.
Best Wishes Scarlett,
SIQ.