They reproduce sounds which could not be captured when recording. For example, the noise of bullets whizzing past and ricocheting are added after filming because no bullets were actually fired and therefore no sound was captured. These sounds can be computer generated or captured seperately from a real event.
Quite often Foley sounds are generated by the Foley artist with no connection with real events - e.g. the clich�d coconuts for horses hooves, a wobbled metal plate for thunder etc.