Looking which way? If looking upstream, towards the start, then it must be the Tower. Can't think of anything like that on the South of the river; Southwark Crown Court is a) reddish and b) nobody's idea of a mansion and c) lacking round turrets, but apart from that, it is more or less in the right area, looking downstream !