Once you move abroad you will be out of juristriction and they will not pursue through the courts. It is likely after a time that this will then be written off. However if you return to the UK within a certain timescale (poss 6 years though not sure) they may write it on again if they are notified you are back (notification being either through benefits or other claims).
Personally, if I knew they were paying me extra and I still accepted it I would feel morally obliged to pay it back. On the other hand, I know of someone who has had dealings with these people and they have on occasions sent two conflicting letters on the same day, one stating that they owed him money, the other stating that he owed them money. After many telephone calls they were unable to supply sufficient evidence/statements to support either letter and he hasn't heard anything since, that was approx 6 months ago and the sum was around �650.