Savings accounts are always a good idea. We opened a small account for my step grandson, but also bought a silver box for keeping his birth certificate in (we had to check with his mum that she didn't already have one, though). Most jewellers have a lot of silver items for christenings, so it's worth a browse.
A bit late now unless you can get back dated coins but when my new granddaughter was born in February I bought her a set of mint condition coins dated this year (the year she was born in). They are in a presentation pack with a 'new baby' book from Westminster coins (I think it was) and cost £26. I did the same for my older granddaughter when she was born. Failing that a set of coins dated this year.