Did the first husband die or was there a divorce? not sure if it makes a difference though. I don't think the first 'grandfather' is actually related at all, he is just ' grandmothers ex husband' .
If I'm understanding this correctly, your grandmother's 2nd husband is your actual grandfather, i.e. he's the father to one of your parents ??
In which case, I'm in agreement with the previous posts, he's not actually related to you.