On Positive Moderation Decisions

A crucial role of moderators is to decide what content is allowed in their community. Though research has advanced its understanding of the content that moderators remove, such as spam and hateful messages, we know little about what moderators approve. This work analyzes moderator-approved content from 49 Reddit communities. It sheds light on the complexity of moderation by giving empirical evidence that the difference between approved and removed content is often subtle. In fact, approved content is more similar to removed content than it is to the remaining content in a community---i.e. content that has never been reviewed by a moderator---along dimensions of topicality, psycholinguistic categories, and toxicity. Building upon this observation, I quantify the implications for NLP systems aimed at supporting moderation decisions, which often conflate moderator-approved content with content that has potentially never been reviewed by a moderator. I show that these systems would remove over half of the content that moderators approved. I conclude with recommendations for building better tools for automated moderation, even when approved content is not available.