If its a formal caution it will be recorded by the police, kept on record and may be cited in any subsequent court proceedings. You would have to sign two written copies of the caution & you keep one.
As I understand it an oral caution (i.e. informal) should not be recorded for the purposes of being cited in court (although I should think there still has to be some kind of record of the caution to avoid someone being repeatedly cautioned).