What your seeing is most likely a problem with your ISP's Contention Ratio. This is a rating of how much the ISP over-sell the bandwidth they have available. If an ISP has 10MBps of bandwidth available and it sells 1MBps to 100 customers, then it has a contention ratio of 10 to 1 (10:1). It works on the idea that all customers won't be using the internet at the same time, but when things get busy i.e. during peak usage during the day, you'll start to notice speed falling as you can't draw your full 1MBps from the pool. Some ISPs can operate at contention ratios as high as 48:1.
Try the internet connection test in the middle of the night (3-5am) when usage will be down, if you're still having speed issues, contact your ISP. Also try a speed test at www.speedtest.net and use the closest server. If it's still only 500kBps then it's definately an ISP matter, if it's faster teh slow-down could be at your company's end.
Also you could try getting a contention free connection, though this can be expensive some times.