I am often asked this question, and I must say it irritates me.
The most memorable time was last year in the park when walking the dog:
Stranger that I was on nodding terms with as she had a dog: Cold today isn't it - bet your not used to the temperature where you are from.
Me: Well, I live in the house over there, so yes, I am quite used to it.
Stranger: No, before you came here
Me: Well, Northampton got pretty cold.
Stranger: Well, I mean where you were born.
Me: St Albans has a pretty similar climate, despite it being in Hertfordshire. Maybe not quite such cold winds as the ones here on the east coast.
Stranger: Well, where you parents were from then.
Me: Ah, Staines in Middlesex. Don't really know that area as I have never been there. Is it much hotter there?
Stranger: Well where were your grandparents from?
Me: They came from Sri Lanka.
Stranger: Bet it's hot there.
I agreed with her.
Why I get irritated is because I am not seen as British but as a foreigner due to the colour of my skin.
Incidentally, I don't see it as racist, just ignorant.