Microsoft Copilot security directory

Content Moderator

Published by Microsoft

High risk

Content Moderator is a Microsoft Cognitive Services product which provides machine-assisted moderation of text and images for potentially offensive or unwanted content, augmented with built-in human review tools

Security assessment

Plutonium assessed this connector for permissions, capabilities, and security-relevant behavior.

  • Reads your private information: This connector can read data the connected service holds about you - documents, messages, contact lists, source code, customer records, or saved profile details - and pass it back to the AI.
  • Can change or update your information: This connector can edit, rename, overwrite, or otherwise modify records in the connected service. If a request gets manipulated, your data could be altered without you noticing.
  • Can execute code or queries: This connector can run arbitrary queries or code against a database or execution engine. An AI-generated query could be manipulated to exfiltrate, corrupt, or destroy data.
  • Sends your data to outside companies: Anything you share with this connector flows to a third-party service. That company sees, stores, and may use the data according to their own policies.

Available capabilities

This add-on exposes 8 tools or capabilities.

  • Check if an image contains racy or adult content
  • Create Reviews for Reviewers in your moderation team
  • Detect Language of a given text input content
  • Detect profanity and match against custom and shared block lists
  • Execute desired workflow in your team to evaluate image or text content
  • Find faces in an image content
  • Match an image against one of your custom image lists
  • Return any text found in an image for the specified language

The interactive security report will load automatically.