Your current employer would have the right to sue you for any loss of income resulting from your early departure. For example, let's assume that you're getting £6 per hour. When you suddenly walk out your employer can't immediately replace you so, for the 3 weeks when you should have been there, he has to get an agency worker at £12 per hour. Assuming that you work a 40 hour week, that's an extra £720 the employer will have to fork out. He has every right to insist that you pay that money, taking action through the courts if necessary.
In practice very few ex-employees are sued by their former employers under such circumstances, but it's always worth remembering that it can happen. (If a senior worker in the finance industry walked out when he should have been finalising a major contract, his old employer might lose millions of pounds. The employee can then sue the former employee to recover that money).
Chris