Do you not know that a little yeast leavens the dough? Clear out the old yeast so that you may become a fresh batch of dough. Beginning chapter five, Paul begins his discussion and/ instruction about the moral decay with which the Corinthian community was becoming synonymous. And the first one that he had to deal with is incest, a sin so grave that even pagans find abominable. And as if such acts were not reprehensible enough, the community drew pride, not shame or sadness, from them (relativism/liberality as a mark of maturity was one of the teachings fronted by the false  wisdom that Paul had attacked earlier in the letter: there was no better way to express this than  freedom from moral laws that were seen as holding people back). It was a reaction that clearly indicated to Paul that the community still held fast even to those gentile practices that were not compatible with the Christian faith. The community had to make a choice: they were to fully commit themselves to the Christian faith or opt out. They couldn’t have it both ways. 
Paul was angry at the larger Corinthian community for condoning those among them who were not true to the faith they had professed. The larger community had the moral responsibility of correcting such individuals in a brotherly way. And even though the community’s correction might appear a bit harsh (expulsion), the community needed not to be afraid, for their correction was for their wellbeing (cf. I Cor 5:5: deliver him for the destruction of his flesh so that his spirit may be saved). Otherwise, the larger community was putting itself at risk of getting consumed by the sin.