it does take longer, yes.
this is because IE is built into windows itself; it's actually part of the operating system (some parts of it, anyway). so when you click on the IE icon, it doesn't have to load the whole thing.
firefox, like most other programs you download and install, are not part of windows. so it needs to load the entire thing, and thus takes a little long.
also, due to the way it's been programmed, it is a little slower than IE. but this is negligible.
if you have a really fast computer, then it will load up very fast indeed. on older, slower computers, it takes more time (obviously). and of course, if the computer is doing anything else when you ask it to run firefox, it'll take longer too. finally, extensions you add to firefox also make it take longer to open.