As a Guild master of a guild that had over 600 members, I had to do this lots of times. If you are unable to kick people when the time comes, then you may not be the right person to lead the clan/guild/team.
There was a time though I had to kick a member for things he would do, say, and react. After I kicked him he was making sure no one would ever want to be part of our guild. It really sucked as he made our reputation go down a lot. Eventually though he became known for just pissing people off and trolling, so our reputation came back.