noxlumos's post is, as usual, a very good and wise one, however, i'd just like to add a few things.
My daughter is Autistic, and as such, I have had some experience with Aspergers sufferers also, and know how much hard work they can be, can you talk to your Social Worker?, because it just possible that this could be caused by the stress and worry of looking after him, and they may be able to qualify you for some respite care, and if your already getting some, a few days more.
The thing is, if your son can go into respite for a few days, and you get someone, (family etc) to look after your daughter, you and your partner can go away and totally relax for a few days.
But you do need to talk to someone as well as you partner about this.
I really wish you, your children, and your partner all the best for the future.