How should frontier models be reviewed before release?

So then let me make this point. So Sacks, there was this argument that DEM has put out for a FINRA style regulatory system, where industry participants that understand technology, move quickly and are aligned with commercialization, actively review each other's systems and each other's models via an industry consortium that hires and assigns scientists to work on this stuff, and they're just doing it versus having it be owned and managed by the government and create some regulatory capture scheme. But then this week, you were pretty against the FINRA system in your tweets. You seem to be supportive of it the first time around when DEM has originally published it. It seems to me that the proposal from DEMIS that Dario signed up to, Sam signed up to, Elon all signed up to, you're now against, but that could be a solution if there is no genuine concern here, given Chamath's points. The industry can police each other.

Seek to 12:01

Having spoken to Demis at length about this, I think that the general framework is SRO in the right structure. However, FINRA is often used as the example of an SRO, simply because it is the most well-known SRO. But you are correct. I think that there's a mechanism by which this thing needs to operate that is distinct from any other SRO. But the idea being the industry gets together, and to Chamath's point, they get to look at each other's models, they get to police it, they get to have independence, but they don't get to be controlled by government.

Seek to 15:16

Okay. So now what you're saying is very interesting, which is you mentioned the term self-regulatory organization or SRO, and there's many different models. And one of them may work. And that's been my point. The one I think is the most appropriate for this circumstance to be something like the MPAA, which does not report to the government at all, and was formulated by the motion picture industry in response to the Hays Code. And what the government said is, get your act together, guys, or we're going to have to do it for you. And what the MPAA did then was promote standards that then forestalled more intrusive, heavy-handed government action. So that is basically the model that I said I could get behind. But is that the model that Dario supports? No. Dario and I think Demis wants a FINRA, which is basically this rule-making regulatory agency that would not only promulgate standards, but also do pre-release model testing, okay? And it would then report into the government and then officially be enshrined in law. And the reason why everyone says, well, Elon's behind her, I've talked to Elon about this. Elon does not support a FINRA for AI. What he could get behind is an MPAA for AI. And that's my point, is Demis wrote something very high-level.

Seek to 13:40

The fact that Dario could want and has called for, including in very recent blogs, a regulatory apparatus modeled after the FAA, and that that is the right model for AI, in my view, is completely discrediting. And yes, we will get smoked by China because they will not subject all their companies to these constraints. Now, Okay, so let's play this out. Let's play this out. Go back to your Steelman for a second. Just because I believe that we should not have a DMV for AI does not mean that I'm against AI safety. So let's talk about some of the things that can be done. So first of all, I think it's good that these frontier labs take safety seriously. Sam just came out with a post, I believe yesterday, saying that actually they're going to slow down the release of some of their frontier models because they felt that their safety stack was not keeping up with the capability. Okay, that's responsible. By the way, do you know why I think they're doing that? Because they have huge and ample liability to take responsible steps. Because if they don't, they're going to be subjected to all sorts of product liability lawsuits. Look at what's happening to Meta right now. They're getting sued because supposedly their product led to social media addiction, and they're being sued for over a trillion dollars right now. This comes on the heel of them losing a $6 million jury trial in Los Angeles where a plaintiff sued them for creating her body image issues. We are seeing massive product liability lawsuits against big tech companies like Meta and Google. If OpenAI and Anthropic are smart, and I think they are, they do not want to be next. So these companies have ample reasons for liability protection, not to just race out their product, okay? Safety is part of what they have to get right. Now, how should the industry coordinate with each other? There are tried and true ways of doing this. Companies in the industry publish papers, they publish technical papers, they publish blogs. These are good ways of sharing best practices. They attend symposia and conferences and they present at these things. They do requests for comments as part of open standard setting exercises. There are ways of the industry collaborating on safety in very productive ways. But what is the common denominator of everything I just said? They are all open. They are not in secret. And that makes those judgments contestable to the point that Chamath made earlier about we need to see the thinking behind this. And what I'm against is these companies meeting in secret, whether it's part of some sort of like FINRA or otherwise, to set standards that are not transparent, that are uncontestable. And you know what's going to come out of that? Regulatory capture and anti-competitive behavior. So there are ways of doing this that will protect the public that are open and transparent. That is what we should lead towards.

Seek to 17:33

Okay, okay. So you're bringing up, it's an exceptional, going down an exceptional path. I love this. There is one thing that you must do if you believe that this is true, to further steelman this thing. You cannot obfuscate the thinking tokens of a model then. What do I mean? If you just compare open source models versus the closed frontier models, there is no way for us to understand what the thinking tokens are, meaning you give it a prompt, you get back something, but all the stuff in the middle is obfuscated, Claude obfuscates it, okay? So do the OpenAI models. And I would say, if I was really afraid, the first thing I would do is try to find a way to unobfuscate these thinking tokens so that you could have a third party actually legitimize what you're saying. Because what's interesting to me is I haven't seen the same doomerism from any open-source model company, including NVIDIA. And the thing with the open-source models is you can actually see it thinking, so you can see its misalignment in real time. And so instead, we have to basically agree to their interpretation of tokens that we can't see, that only they can see, that they can interpret how they want. And without judging whether it's right or wrong, I would say give them the benefit of the doubt that they're trying to do the right thing. Make the bloody thing available to other people then. Let somebody else with some amount of judgment as well, sit in the room and steel man it with you.

Seek to 10:24

You will not see it happen. And I'll tell you why. Just again, the technical details matter. If all of a sudden competitor A, B and C could see every single trace and span that your model generated as an artifact of going from input tokens to output tokens, you're essentially giving away a lot of your recipe. And I just don't think that if Anthropic isn't willing, even if I'm an API customer, to share my thinking tokens from my company with me, they're sure as sh** not going to share it with OpenAI and anybody else in a room.

Seek to 20:21