Simple answer: No
For example, the attacking team might be taking a corner. Just as the ball is played, the referee spots a defender nudge a forward in the back, in order to make sure that he'll be off-balance should the ball happen to float towards him. It's definitely a penalty (for intentional pushing) but, since there was no clear goal-scoring opportunity it doesn't fall into the 'serious foul play' category (which would result in a red card). Similarly the referee would almost certainly think that 'ungentlemanly conduct' (which would result in a yellow card) would not be appropriate either.
Chris
PS: I chose that particular example because it's one I remember from refereeing a local semi-final. I was the only one on the pitch (other than the two players giving and receiving the nudge in the back) who was aware of the incident. (Even the FA Premiership official, who was running the line at that end of the pitch for me, didn't spot it). I had no hesitation in awarding a penalty but I never even considered cautioning the defender, or sending him off.