What a dreadful situation for you - I feel deeply for the pain you must be suffering.
First things first - you are only the biological offspring of this man - he has no links to you other than that. Ther person you are is down to your upbringing, and that has obviously been in an atmosphere of love and understanding, and that is what you must hang onto.
Please try not to be hard on yourself about the circumstances of your conception. You are an individual, thoughtful caring intelligent woman, and the origins of your life are not as important as what you and your family have made of you as a person, you must focus your mind on that, and push away the thgouhts of this man, it will only hurt you, and solve nothing.
With regard to seeing your brothers - that is something you need to take a step at a time, and find your way through an emotional minefield, so tread gently, and be nice to yourself all the time.
I hope you reach the peace you are seeking.