Dear fruitsalad,
I lost my father two weeks ago. Like you and your father, we never felt easy about physical contact, however, in the last couple of months we managed to get past those inhibitions. On his death bed he summoned up the strength to hug us all and whisper goodbye. I cherish that and consider myself extremely fortunate to have had that moment.
Despite that, I still have an "I wish I had", which was that we would talk, that we could express love verbally. This never happened and I will never get over that feeling, it should have been so easy, and now it is too late.
I think my point to you is that everybody is going to have regrets that will haunt them alll their lives, and whilst it will prey on your mind and hurt, I try to believe that what matters is not what you did or didn't do, but the times you had together. Love between a father and child is unconditional.