I know that it upsets you that he's with her, and yes, she may not be a very nice person. But if you want to continue contact with your brother, you must accept her.
One day, he will probably be forced to choose between you. And he WILL choose her, love is blind, deaf and dumb.
The only thing you can do is accept her, be nice to her (even when she isn't to you) and hope that he doesn't feel too pushed away from the family for good.
Otherwise, whether they spilt up or not, he will not come back to the family for fear of repitition.
And also, how do you think she feels?
She might actually be quite a nice person, but you have got a bee in your bonnet some 4 years ago, and won't let go.
She's probably not very nice to you because she thinks you don't accept her and don't like her.
I know all this from personal experience, I dislike my fiances family a lot. As they did not accept me when me and my partner first got together. Now they try to be nice, but I can't let go. I try to avoid them, and try not to go round there when I can.
I had to stop my fiance choosing between me and his mum, and he was planning to give everything up for me. But I didn't feel it right, as you should always be close to your parents in my opinion.
And please, if you don't like her, don't start telling him. How on earth would you feel if the person you completely adored, couldn't live without was hated by your family, and you were reminded everyday. It's very harsh you see?
Leave them, and if they break up, just comfort him, and don't hold a grudge.