Sorry I don't actually know Leicester but I went to Branston Water Park which is near Burton-On-Trent a couple of years ago. It is less than 40 miles from Leicester and has a wealth of birds, I saw a tree creeper there.
I wouldn't have counted chaffinches as rare birds, I think they're common up and down the country.
We were in Leicestershire at the beginning of the week, and my Father in Law was commenting on the lack of bird types lately. It seems it's just blackbirds, sparrows, starlings, pidgeons and doves making the rounds. Perhaps with all the new estates going up, it's driven them further into the countryside? Abbey Park used to be quite nice and open, you could try there perhaps?
none of the above are what I would call real rarities. A quick google on country parks found the following
Brocks Hill Environment Centre & Country Park - 3.3 miles SE - Washbrook Lane, Oadby, Leicester, LE2 5JJ - 0116 271 4514
Watermead Country Park - 4.4 miles N - Severn Trent Works Fillingate, Wanlip, Leicester, LE7 4PF - 0116 267 1944
Bradgate Country Park - 5.4 miles NW - Bradgate Park, Newtown Linford, Leicester, LE6 0HE - 0116 236 2713