Yes, they plug into the USB port (preferably USB 2.0) or into a Firewire port (some come with both). No, they don't take up memory. Only problem is with some laptops not being able to supply enough power to the USB port. Some come with an extra lead for this. The larger ones have a mains unit. Nicest ones are from Freecom 2.5inch, small, portable and up to 80GB.
Further to wideboys reply Get your self a powered USB hub.
External HD's dont take up any memory on your CPU but as with all USB devices they will use up power resources on your PC if you plug them in directly. If like most people you have a USB mouse, keyboard, printer, Digital camera etc, etc they are all competing for power from your system. This can lead to a fairly obvious slow down. A powered USB hub get it's power straight from the mains and you can then route up to seven USB devices straight into one USB on your CPU.
Tidy