Symbian is the name of an operating system that derives from the Psion series of PDA's that has since been discontinued.
Basically it's the software that runs the phone. Nokia places a bit of their own software front on top of it. The N series phones are all Symbian, as well as others like the now pretty old 6600.
The advantage of it is that it's more like a proper little computer (hence them being called smartphones), and so you can have better file management and install your own little programs on there much more easily than with custom operating systems that run the other phones.
The disadvantage is that because it's more powerful the phones generally run a little slow (I find at least); certainly not as brisk as the non-Symbian Nokia phones.
If a manufacturer wants to make a smartphone with these sort of capabilities they usually choose either Windows Mobile or Symbian.
Nokias are generally classed as series 40 and series 60 phones. The series 40 are their standard phones with their own OS; the 60's run on Symbian.