Sensible question.
Yes, all birds lay eggs.
Yes, all birds have sex.
However, not as we know it... In most birds, the male has no organ, and is constructed much like the female. The male mounts the female, twists his tail beneath hers, and the sperm is sort of sucked into the female. Exceptions are waterfowl (ducks, geese, swans), and the ostrich family, which have a penis not unlike a mammal's -- some of these are quite well endowed.
Also, the testes are not external, in a bag like mammals', but inside, up against the spine near the kidneys.
Only one ovary (the left) develops in the female. If this goes wrong, the other rudimentary gonad can develop -- into a testis... She then becomes a fully functioning, fertile male!
In birds it's the female who has the equivalent of the Y chromosome, with the male having a pair equivalent to the Xs of female mammals.