Discord bans teen courting servers and the sharing of AI-generated CSAM

Discord has updated its coverage meant to guard kids and teenagers on its platform after reports got here out that predators have been using the app to create and unfold little one sexual abuse supplies (CSAM), in addition to to groom younger teenagers. The platform now explicitly prohibits AI-generated photorealistic CSAM. As The Washington Post not too long ago reported, the rise in generative AI has additionally led to the explosion of lifelike pictures with sexual depictions of kids. The publication had seen conversations about using Midjourney — a text-to-image generative AI on Discord — to create inappropriate pictures of kids.

Along with banning AI-generated CSAM, Discord now additionally explicitly prohibits every other form of textual content or media content material that sexualizes kids. The platform has banned teen courting servers, as properly, and has vowed to take motion in opposition to customers participating on this habits. A earlier NBC News investigation discovered Discord servers marketed as teen courting servers with individuals that solicited nude pictures from minors. 

Grownup customers had beforehand been prosecuted for grooming kids on Discord, and there are even crime rings extorting underage customers to ship sexual pictures of themselves. Banning teen courting servers utterly might assist mitigate the problem. Discord has additionally included a line in its coverage, which states that older teenagers discovered to be grooming youthful teenagers can be “reviewed and actioned beneath [its] Inappropriate Sexual Conduct with Kids and Grooming Coverage.”

Except for updating its guidelines, Discord not too long ago launched a Family Center tool that folks can use to regulate their children’ exercise on the chat service. Whereas mother and father will not have the ability to see the precise contents of their children’ message, the opt-in device permits them to see who their kids are pals with and who they discuss to on the platform. Discord is hoping that these new measures and instruments might help hold its underage customers secure together with its previous measures, which embody proactively scanning pictures uploaded to its platform utilizing PhotoDNA.