The earliest-recorded written version of this appeared in Sinclair Lewis's 'Main Street' published in 1920 as "no skin off my ear". Since then, it has appeared as "no skin off my.....", with the dots standing for just about any part of the human body! The "nose" version first appeared in 1925. The answer to your question is probably, therefore, that Sinclair Lewis 'invented' it.