The "Big Bang" was a name given to the theory by someone who didn't like it much, namely Fred Hoyle. It's a bad name but has somehow stuck. I don't understand most of it myself but even so it's not really a "Bang", but more of an expansion. Space and time were all gathered at a single point, and just grew outwards. Lots and lots of energy released, and crazy stuff went on, the full details of which are still a mystery to us all.
But the reason the theory holds is because this event left behind a lot of traces: an expanding universe, lots of background energy that shows up as microwaves, and other things I'm not aware of, all of which fit the theory well. There are still gaps, and holes, and things to be sorted out, but the overall picture is very convincing and as a result the general gist of the theory is almost universally accepted these days.