As long ago as the 1500s, the word 'pinky/pinkie' was a childish adjective simply meaning 'small'. Thus, one might have 'pinky eyes' or a 'pinkie hole' in one's sleeve. In the early 19th century, it came to be used as a noun and most specifically for the little finger.