It usually means waiting for something which seems bound to happen, though it may not. It derives from an old joke about a man coming into a boarding house late and taking off one shoe and dropping it heavily on the floor, waking up someone sleeping below. The person below then waits indefinitely for the other shoe to drop, not knowing that the man above, realising what he had done, has taken off his second shoe quietly. The "other shoe" saying has similarities with the expression "the dog that didn't bark" which has its origin in a Sherlock Holmes story where what the great detective found curious was that a dog did NOT bark when it might have been expected to.