The real question is whether these classifiers can find all the dogs that Dave has at home.

I distinctly remember being at Meta in the wake of the Christchurch massacre, when horrific videos were circulating across Facebook without end. The technologies we had for being able to accurately classify videos just didn't exist. [1/n]

Today we're releasing Coop 1.0, the world's first free, open source content review & enforcement system any org can self-host and build on. For the first time, any org, whatever its size or budget, can review, act on, and report CSAM end to end, for free. roost.tools/blog/coop-1-...

Today, in a single day, @dwillner.bsky.social and I had meetings with people in England, Turkey, NYC, SF, Argentina, and Australia. Very inspiring to see four continents of folks all using Zentropi and united in the earnest work of building a better internet.

Can your LLM actually follow your content policies, or will it just revert to its own static training? Today we're introducing a new benchmark that we call policy steerability that tries to measure this concept: blog.zentropi.ai/introducing-...

Very cool! @julietshen.bsky.social's independent tests show that our new model CoPE-B cooks :-) Direct link to her results: github.com/julietshen/c...

The ROOST Model Community is growing! Today we welcome Zentropi's CoPE-B-A4B, a bring-your-own-policy model that's got the power of 25B but runs on only 4B active parameters. It's a fast, low-cost model that can be used on its own or as a first pass before larger models! roost.tools/blog/welcomi...

New in the ROOST Model Community: zentropi.ai 's CoPE-B-A4B is here. roost.tools/blog/welcomi...

a few use cases I can think of for CoPE-B (and BYOP models): if you're a platform that's ok with NSFW role play but not age play, you can create a custom CoPE model that looks just for that. Many free text classifiers come with baked-in ideas of morality that might not fit your community

Exactly right. Speech shouldn't be ruled by the platform hegemons. Your rules should rule.

We are THRILLED to announce that @roost.tools is jointly releasing CoPE-B with the Zentropi team. We believe that everyone should have access to openly licensed models designed for safety use cases that can be tuned to their community's norms. Join us at RMC office hours next week to learn more!

Super pumped to release CoPE-B, our latest policy-adaptive content classification model. It delivers frontier-level accuracy in a self-hostable package that's orders of magnitude cheaper to run-- opening up new possibilities in trustworthy platform design. Details: blog.zentropi.ai/meet-cope-b-...

@samidh.bsky.social and I are releasing CoPE-B today, the next version of our policy-adaptive content classifier. It delivers at-or-better-than-frontier classification while being self-hostable, faster, and cheaper to run. Full writeup with benchmarks at blog.zentropi.ai/meet-cope-b-.... 🧵 1/9

Very excited to have this public. The @oversightboard.bsky.social's use of Zentropi to study child marriage content on Meta platforms was very cool to support: blog.zentropi.ai/how-the-over...

Check out how the @oversightboard.bsky.social used Zentropi to better understand how child marriage-related content manifests on Meta's platforms. Fantastic example of how advanced content labeling technologies can strengthen both our online and offline world. blog.zentropi.ai/how-the-over...

It has been incredible partnering with character.ai since the very start of zentropi.ai. We're excited to share some details of that partnership with this case study. Anyone creating AI-powered systems might find it interesting! blog.zentropi.ai/how-zentropi...

In 2017, it took us months to define “political ad” at Facebook. Recently, I built two political content classifiers in an afternoon using Zentropi AI created by @dwillner.bsky.social and @samidh.bsky.social Why AI content moderation is good, actually — in this week’s newsletter 👇

One of the things we've been thinking about a lot at Zentropi is: what happens when AI agents need to make judgment calls about content — not humans reviewing a queue, but agents acting autonomously?

There's a major gap in content safety tooling: classifiers typically only score complete text. When you're working with generative AI, "complete text" means the user already saw it. That's too late. So we built a streaming classifier that we're releasing today! Here's what we did and why. 🧵...

If you're not using either tool yet, now's a good time to try both! Zentropi's Community Edition is free and gives you unlimited labelers. Coop is fully open source and runs on your infrastructure. :D

Zentropi is now integrated into Coop, @roost.tools's open source moderation platform. You can write a content policy in plain English on Zentropi, plug it into Coop as a signal, and have a moderation pipeline running in minutes.

Dave Willner, who led trust and safety at major tech firms and has cofounded a company that is developing an AI content classification platform, says LLM-driven technology can now accomplish classification at the scale necessary for moderation on large platforms. That has substantial implications.

I can has cats.

Just shipped Zentropi's most requested feature: image classification! Now analyze images against your own policies, at scale. To power it we built cope-b-12b, a new multimodal model w/ native vision. Check out the cat detector we made in < 1 min. 🐱 blog.zentropi.ai/zentropi-now-labels-images/

If you are looking for a technical description of how X rots your brain, look no further than their github post on the 'X algorithm'. It is pure, unadulterated behavioral engagement maximization that amplifies the very worst human impulses. github.com/xai-org/x-al...

Why are we just giving away all our secrets? Well, it is our hope that it helps the ecosystem further advance the state of the art in policy-steerable content classification, which is foundational to a more trustworthy internet.

Dave just published a Zentropi labeler that can precisely identify requests at prompting an AI model to undress a person in a photo. The tools exist to easily deal with this problem -- platforms just need to choose to use them. If you are the developer of an AI system, please use this guardrail!

This was such a cool experiment that I created a Zentropi labeler with a simplified version of the authors' Partisan Animosity criteria. Now anyone can experiment directly with using this labeler to try to reduce the temperature of affective polarization in their feeds. zentropi.ai/labelers/b30...

We just wrote an in-depth post about Toxic Content labeling. It presents a new way of defining toxic speech online-- and illustrates the importance of observable features for accurate language model interpretability. Would love to hear how YOU define toxicity, too! blog.zentropi.ai/observations...

Awesome to see how this is already being used! One of the most useful aspects is that the published policies show what it takes to write content rules that can be accurately interpreted by language models. We hope this can be a boost to the broader content policy community.

This was a fun launch! It turns Zentropi into a Github for Content Labelers. You can share content policies with others and build off each other's work. It's the easiest way of deploying a fully customizable classifier. Check out the policies @dwillner.bsky.social created at zentropi.ai/u/dave

Content policies are usually private, one-off efforts. You build yours, I build mine, we don't share much about what works or why. This makes sense given products can (and should) set different policies based on their communities, but it leaves us reinventing the wheel. 🧵 1/5

We got really positive feedback on the TrustCon workshop we ran on writing good content policies for LLMs...so we're doing it again! If you're interested go sign up here, so we can start to figure out timing: forms.gle/tj7vf7ng8n7R...

This response to the Raine tragedy from OpenAI does something remarkable: it has the humility to acknowledge that a *product failure* led to real-world harm. Despite horrific circumstances, it has a rare degree of honesty that I wish tech companies would show more often. openai.com/index/helpin...

We are opening up Zentropi.ai to everyone today so that anyone can build their own content labeler. What started as a crazy academic idea 2 years ago is now a real thing that companies are using in production to safeguard their AI-powered systems. Give it a shot! blog.zentropi.ai/zentropi-bui...

Don't take our word for it! Go kick the tires at zentropi.ai and build your own content labeler (no subscription required!)

@mmasnick.bsky.social I have a bluesky demo for you that you might want to see :)

Just tested this on a few that I know Reddit’s existing Hatred & Harassment automation has blindspots for; it built a focused, accurate labeler in under 10 minutes / a dozen examples, & the human readable criteria it built could be dropped into a training manual / erratum / used to build a regex

So excited for #TrustCon this week! We will be publicly unveiling Zentropi, a platform that helps people instantly build their own content labelers. We'll be opening it up for early access and open sourcing the underlying language model we trained for the task so that it is accessible to everyone.

I expect @dwillner.bsky.social to run around like a maniac again at #Trustcon this year as he shows off Zentropi -- our platform that makes it simple to build your own CoPE-powered content labeler.

A year ago at #TrustCon, I ran around like a maniac showing people something on my laptop. We'd just gotten CoPE - our policy interpretation model - working. It felt like a huge achievement, validating our ideas about LLM-powered labeling. 🧵 1/7

Take back your attention.

The splinternet accelerates. If this stands, look for more countries in 2025 to ban Facebook, Instagram, YouTube, etc. out of fears of American surveillance. www.bloomberg.com/news/article...

If someone on my team at Meta had ever said these kinds of words, I'm pretty sure I'd have had an obligation to notify HR. But maybe times have changed and "masculine energy" is now part of the performance review rubric. www.bloomberg.com/news/article...

Will Meta's retreat from proactive/automated enforcement of its community standards lead to Instagram and Facebook becoming deeper cess pools? Well, it's complicated and the devil will be in the details. Let's look at the data and see what questions we should be asking... 🧵 1/12

Folks are mostly missing the forest for the trees re: the recent Meta announcement. Specifically, the focus on the move from fact checking to community notes. There two different changes that are a much bigger deal that have received less focus. 🧵 1/11

Here's my "fact check" on Meta's announcement that it is terminating its fact checking program. The TL;DR: Pay more attention to product changes than to political pandering. THREAD...

@samidh.bsky.social and I have made a special-purpose language model for content classification that matches GPT-4's performance, but is much smaller/faster. We've got a demo up on HuggingFace and weights are available to partners today - check out the link for more details, including how to help!

To complement the article @dwillner.bsky.social and I wrote on using LLMs for content moderation, we created a Content Policy Compiler GPT to transform your content policy doc into one that's more accurately interpretable by LLMs. Very rough, but give it a shot!