If you're genuinely self-employed then (unless you've got a sub-contractor's contract which states otherwise) your contract can be terminated at any time and for any reason.
To be able to make a valid legal challenge you'd first have to be able to establish that you were actually an employee, and not self-employed. (That's tricky and you could end up with you having to pay tax and National Insurance which you've so far avoided). Then, since employees can normally still be dismissed without any reason during the first 12 months of their contract, you'd have to show that you had been subject to racial discrimination, which is unlawful at any time. (Under the Equality Act 2010, the definition of 'race' includes nationality).
I can't see any way in which you could do that but you might like to speak to someone on the Equality and Human Rights Commission's helpline to check the position:
0845 604 6610
Chris