It is easy for people to say get out and finish it now, its just not that easy , Just have a no holds talk with him ,tell him how you feel and how miserable he is making you...... and is he going to improve his behaviour, if it does not change then you can look at progressing it to the next stage ,
Are you sure you have not got chronic fatigue ??
Btw , my wife is a kept woman , (shud be kept in a cage!)and i often say to her , how can you spend so much money ?? she just tells me "you should come out with me I will show you how I spend Your money" .... ;-)