The simple rule is that normally fruits grow on bushes and trees above ground and veggies grown below ground in tuber form or otherwise, thought there are exceptions to these rules.....So olives are fruit.
Janetex is right - if it has a seed, then it is a fruit. Thus olives are fruit, tomatoes are fruit, marrows and courgettes are fruit, cucumbers are fruit.
Rhubarb, however, is a vegetable, as is spinach, cauliflower, cabbage etc.