The earliest written recorded use of the word 'shoestring' to mean 'a small amount' appeared in the 1880s. Oddly, though, that reference goes on to read, "as the saying went". Clearly, then, the writer realised it was a kind of proverbial spoken usage for some time prior to that.
The earliest reference to a 'shoestring budget' did not appear until the 1970s. That was in the South Wales Guardian.
Thanks TheWinner and Quizmonster. I heard the term on TV last night and it just got me thinking about the origins of it,I have heard it dozens of times before but never really thought about where it came from.