AI Moderation APIs - CometChat Docs

The Moderation API provides endpoints for managing content moderation rules and keyword lists. You can define rules that automatically detect and block inappropriate content in messages, including text profanity, AI-based image/video moderation, and custom keyword filters.

Key Behaviors

Default rules cover text profanity, AI-based image/video moderation, custom profanity, contact details, and email content.
Text and custom profanity filters scan for words (including regex patterns) and drop the message if a violation is detected.
AI-based image/video moderation scans attachments and discards the message if a violation is found.
When a rule is violated, the message is blocked. Optionally, the sender can be kicked or banned (in groups) or blocked (in 1-on-1 conversations).

How Moderation Connects to Other Resources

Messages — Moderation rules are evaluated against Messages in real time before delivery.
Groups — Violations in group messages can trigger member kicks or bans via Group Members and Banned Users.
Blocked Users — Violations in 1-on-1 messages can trigger a block on the sender.

Available Endpoints

Operation	Description
Rules	Create, read, update, and delete moderation rules
Keywords	Manage keyword lists or regex patterns for text moderation
Blocked Messages	Retrieve messages that violated moderation rules
Flagged Messages	Retrieve flagged messages for review
Reasons	Manage custom flag reasons
Rule Revisions	Fetch historical revisions of a rule

AI Image and Video Moderation Labels

Label	Description
Violence	Instruments or devices used to cause harm — firearms, sharp weapons, explosives, and ammunition
Alcohol	Drinking alcoholic beverages from bottles or glasses
Gambling	Participating in games of chance — cards, roulette, slot machines
Drugs & Tobacco	Tablets, capsules, or the act of smoking cigarettes, cigars, e-cigarettes, hookah, or joints
Rude gestures	Hand gesture with the middle finger extended
Explicit Nudity	Visible human genitalia
Non-Explicit Nudity	Exposed human back from neck to spine without full occlusion
Swimwear or underwear	Swimwear or undergarments — bikinis, swim trunks, swim briefs, etc.
Visually Disturbing	Extremely thin, undernourished human bodies with severe physical wasting
Hate symbols	Symbols, flags, or gestures associated with extremist or terrorist groups

API Constraints

Need higher limits? Contact CometChat Support to discuss enterprise options.

Rules
Lists
Advanced Settings

| Parameter | Limit | |-----------|-------| | Rule ID | Max 100 characters, no spaces or special characters | | Name | Max 100 characters | | Description | Max 255 characters | | Filters per rule | Max 10 filters | | Conditions per rule | Max 10 conditions | | Rules per app | Max 25 custom rules (excludes default rules) |

Error Handling

Error Code	Description
`AUTH_ERR_EMPTY_APIKEY`	API key is missing from the request headers
`AUTH_ERR_APIKEY_NOT_FOUND`	The provided API key is invalid
`AUTH_ERR_NO_ACCESS`	The API key doesn’t have the required scope

For the complete list of error codes, see Error Codes. For system limits, see Properties and Constraints.

​Key Behaviors

​How Moderation Connects to Other Resources

​Available Endpoints

​AI Image and Video Moderation Labels

​API Constraints

​Error Handling

Key Behaviors

How Moderation Connects to Other Resources

Available Endpoints

AI Image and Video Moderation Labels

API Constraints

Error Handling