i maybe reading this wrong so forgive me if i am .
but i get the impression the reason it failed before was because of the sex " being over before it started " ( to be blunt )
is this correct or were there other things that made the relationship lacking ?
if it was just the " sex " side of things , then maybe you feel you didnt give it a fair go . as a first time can be very nerve racking for anyone especialy when you are " over excited " , as you stated it was a long time coming. Not every experiance is perfect as im sure many women / men if honest can tell you .
if you feel that is correct then maybe you should try again .
however that would be your choice.
im not trying to push any point here but, i feel your feelings for the man you currently have in your life may not be enough for you , as you state yourself that this man makes you happy but does not give you the same feelings as this other man you cannot forget. " just wondering if your being fair to yourself and current partner " again only you would truely know the answer to this.
what ever you decide regarding this issue i wish you luck , you obviously have spent a long time thinking about it , i hope the choice you make is the right one for you all.