Internet Explorer is Microsoft's web browser. Bing is Microsoft's search engine (which can actually be used through any web browser). So it's unsurprising if Microsoft have pre-configured Internet Explorer to open with their own choice of web browser in front of you. (i.e. Internet Explorer is opening perfectly normally but simply with Microsoft's default page shown, rather than your own preference).
What do you actually want to see as soon as you open Internet Explorer? A blank page? Your ISP's home site? The Answerbank's 'front page'? The BBC news site? The log-in page for your email account? It's up to you to decide what goes there!
You simply need to change your home page (which Internet Explorer also uses as your 'start page'):
Open Internet Explorer.
Go to a page that you'd like to set as your home page. (e.g. the BBC news website, the Answerbank or whatever you want)
Then follow the instructions under Option Three here, selecting 'Use this page as your only home page':
http://www.sevenforums.com/tutorials/155721-internet-explorer-home-page-add-change.html
Job done!