Fanny Adams was a young girl who was callously murdered in the 1860's. Her murder caused revulsion across the country and the saying came into use shortly afterwards, initially as an expression of pity. More here.
http://www.hants.gov.uk/museum/curtis/fannyadams/
Indeed, that may have been the intention, New Forester. But given that it came from 1860s it is doubtful. Possibly sweet **** all came from the phrase 'sweet Fanny Adams'.
I've heard that expression said so many times over the years, but never gave a thought to the true meaning. I've just looked at that website Mortartube & how very sad. God rest her soul.