When someone gives serious thought and consideration they are said to be thinking deeply.
This gives the idea that the brain, or memory, is in layers - with instant thoughts being at the top, and things that need thinking about or remembering further down.
So when you say 'not off the top of your head' you are really saying you don't know without thinking hard about it - the answer doesn't come straight to your mind as it is not at the top!