You deserve to be in a loving and supportive relationship - everyone does. The best way to feel good about yourself and exude confidence is not to wonder if your every move is being judged ,and if you are found lacking ,being punished by that person relocating his affections to another party ( or parties). It was not very kind of your husband to bring all this angst back for you to help him sort out. If I was in your position I would question my other halfs motives : did he choose seriously to be with me, or did he choose to not to be in his new situation ( and I was his easiest out).
I would not consent to 'swinging' if you do not have a VERY solid relationship to begin with as it will only magnify any imperfections to breaking point! I do wonder what the upshot will be if he finds that watching you with someone else does not have the desired effect for him, what then? He thought that being with the other person would be a good idea, apparently not. This looks very harsh in print, sorry Sweetie!
I think that yes you will start to feel good and be attractive and sexy to someone, not perhaps your current partner . Who appears to want to reap the benefits of this new improved you, with out accepting his own role in your present state of mind. I wish you all the very best for your future, on your own, or together ,whichever you feel is best FOR YOU and the kids. Love, Sense.