Further to the previous answers, a diploma is a piece of paper stating that a person has proved their competence (either by examination or through practical demonstration), following a course of study.
A degree, on the other hand, is actually admission into a class of membership of a university. It's usually awarded upon completion of a course of study (i.e. a 'taught' degree) but 'higher' degrees (i.e. some 'master's' degrees and all doctorates) are 'research' degrees where there is no formal instruction. Additionally, since a degree is a class of membership, a university can award a degree (without either study or research) to anyone whom they deem to be worthy of membership. (i.e. 'honorary' degrees).
Some religious sects require that their members must not belong to any organisation other than to their sect. In order to comply with this rule, graduates who join the sect are required to write to their Alma Mater requesting that their degree be withdrawn. (i.e. the university has to cancel that person's 'membership' of the university). Sect members are not required to seek the cancellation of any diplomas, because these don't constitute 'memberships'.
Chris