The issue, 20/20, is that naturally people feel concerned when events seem to be out of their control. There is often not too much to think about at home except to brood about the situation.
Three weeks is not necessarily a long time - some employers spend weeks investigating. Though I appreciate it is hard for you, sarfraz, the fact that is taking a long time does NOT mean that the position is proving to be more serious. Some employers are just inept as dealing with these things. Maidup's advice is spot-on as to what you should be doing (plus trying to relax a little, given the 'free' holiday).