There are two relevant points here:
Firstly (and probably most importantly), while your connection is wireless as far as your router, it's wired from there to to the internet. ISPs operate different 'contention ratios', which represent the maximum number of their customers who are effectively sharing a common connection to the internet. (If you want a connection all to yourself you'll have to pay for a 'leased line'. That's what major companies do but it typically costs around £300 per month).
If you're with a 'budget' ISP, you'll be hampered by a contention ratio of, perhaps, 20 to 1. (Some companies use even higher ratios). 'Full price' ISPs usually use contention ratios of between 15 to 1 and 10 to 1. ISPs who target their products at small businesses (such as Claranet, which I use) operate with a 7 to 1 contention ratio, but at a fairly high price.
So the chances are that your connection problems are on the 'wired' side of your router because the evenings are when most families are online.
Secondly, however, there's the problem of general radio interference. All electronic devices emit radio waves. (Put a transistor radio next to your printer, or to your TV, and you'll probably find that reception is swamped by interference). Your own electrical devices, together with those of your neighbours, will be creating interference which your router and PC have to filter out in order to receive accurate data. The more interference there is, the harder it is for those devices to do their jobs efficiently, so the slower the speed of data transfer. (Simply switching off your TV set might, for example, improve the data transfer speed but [ideally] you'd really need to persuade all of your neighbours to turn their sets off as well!).
Chris