Framer AI AgentsDesign and publish professional sites with AI
Guardex
AI-powered real-time content moderation & censorship shield
4 followers
AI-powered real-time content moderation & censorship shield
4 followers
SafeVision is a developer-first, cloud-based visual moderation API by Guardex Technology. Built for platforms handling user-generated content, it lets you scan, evaluate, and filter harmful imagery (weapons, blood, gore, NSFW) with a single HTTP request. Get exact coordinates for auto-blurring. Built on Guardex's security infrastructure, it ensures ultra-low latency safety layers without slowing down your app.
How does the API handle edge cases where context matters, like a medical app showing surgical images versus actual gore content? Curious if there's a way to tune the sensitivity per use case.
@semihkodaruge Hi!
That’s an excellent question. Handling contextual differences (like medical surgery vs. illegal gore) is one of the biggest challenges in content moderation. Here is how SafeVision addresses this:
1. Dual-Model Architecture: We combine a YOLOv8 object detector with an EfficientNet scene classifier. While the detector might spot "blood" in a surgical photo, the scene classifier evaluates the overall context of the image. A medical/surgical setting does not trigger the "violence" or "hostile" context flags in our classifier, helping prevent false positives.
2. Sensitivity Tuning (Thresholds): Currently, API users can choose between different model versions (like Hawk v1 for legacy compatibility and Hawk v2 for high precision) directly via the API parameters. In our enterprise plan, we allow customizing category-specific confidence thresholds (e.g., setting the threshold for "blood" higher for a medical app, while keeping it low for a children's app).
In a future update, we plan to expose these threshold sliders directly on the Developer Dashboard so users can fine-tune sensitivity dynamically per API key!
Thanks for the great feedback!