If you're just having sex with someone you don't like, for the sake of having a bloke around who abuses and swears at you, then it's time to get a grip on yourself - what the hell are you doing, staying with this man - particularly when you don't even leave together? You are just using each other for money or sex. Life is worth a bloody sight more than this. He's insecure so he abuses you so you'll think you're worthless and won't leave him.
Stop this, now - don't let him in your house any more. Just tell him that you don't love him, you don't want to be with him, you'd rather be on your own than with him - which, tbh, is exactly what you are saying. Tell him the truth, for God's sake - stop pussyfooting around with this man you don't even like.