Writing the speech is the hardest part, so you are past that bit!
As advised, speak clearly, and what may sound like speaking loudly to you, but yuou want to carry to the back of the room. Keep your head up.
There is nothing at all wrong with holding notes, so if it makes it easier, print the whole thing out and read it verbatim - leaving pauses for the jokes if you have any.
Remember, you are not a professional, no-one expects you to be a professional, so be sincere, and smile if you can, and it will go fine I promise.
I have only been a Best Man once, but Father-Of-The-Bride three times, and a second go for our oldest daughter coming up, so I feel for you.
I promise it will not be hgalf as bad as you imagine, because anticipation is always far worse than performance - like hang-gliding!
Best wishes, and leet us know how you got on.