You are referring to the Capsicum family of plants. Usually the larger variety are called (bell or sweet) peppers and come in red, orange, green, yellow varieties. They are not particularly 'hot'.
The thinner variety are known as 'chillies' and their strength is very variable.
Chilli powder usually contains dried red chillies and other spices.
Cayenne pepper is pure dried chilli powder.
Gef's quite right, "Chilli Peppers" are what we usually just call a red or green pepper. (or orange or yellow!) Regarding chillies and their strength, a good rule of thumb is this; Colour usually has no bearing on how hot the chilli is, but size in fact does. Generally, smaller varieties of chilli are hotter than the larger ones.