For him to be successfully prosecuted for fraud it would have to be proved that he forged your signature. This is, I believe, mainly a matter of a handwriting specialist comparing your signature with what is on the documents. If that is done & the conclusion is that you did not sign them, then I don't see how the building society could make you liable for any of the money.
I believe that in this sort of situation the lender may have an obligation to make sure the joint borrower (i.e. you) knows about the loan & - if necessary - is separately advised. However, you would need legal advice on this if the lender tries to pursue you for the debt or to take any part of it out of your share of the equity if the property is sold.
Incidentally, are you sure these loans are in your name as well as his? It would be possible for him to take out loans in his sole name which would be secured against only his part of the equity. This would have been easier for him to do if the lender was willing .