You might find that once it has reheated that the pepper has mellowed sufficiently. Adding yogurt will soften the edge of the pepper but it will also dull the other flavours. If you go down the yogurt route do it right at the end though.
I have not tried it myself, but someone suggested to me that adding a (perhaps boiled??) potato into the dish will absorb some of the chemical that gives pepper its heat.