Did you only send the text today? If so, I'd give it longer. I never reply to texts straight away, because I rarely have my phone glued to my side :-)
I do think the "I doubt you would" line was a bit much. I read it like you were almost seeking reassurance. That's the problem with texts though, as the way you mean them may be completely different to how the other person reads it.... does that make sense? :)
If he hasn't replied by tomorrow mid-afternoon, then give him a call. You haven't got anything to lose. x