which option are you using?
If you go to 'make dvd video' (or whatever it is), Nero will convert the avi file to VOB files to burnt o DVD to allow playback in a DVD player. These take up a lot more space than avi files.
In fact, in this format, you can fit 120 minutes onto a DVD rather than being specifically file size.
try downloading winAVI (it's free) and using the option in that to convert to DVD format, then burning these files (either via winAVI or Nero's 'burn DVD files' option). I've found this works sometimes.