Event Replay: Building the Future of AI: From Weak Decisions to Safer Decisions

[00:00:00] Thanks so much, Jason. [00:00:08] My name is Tobias. I lead a team in our intelligence and investigations team. [00:00:14] And what is that? Well, first off, I'm not a scientist. I'm going to break it down a little bit more to operational issues. [00:00:21] So the intelligence and investigations team is pretty much a very downstream team in our safety stack to ensure safety of how our models are deployed in real life. [00:00:33] So how are people using our models in the world and particularly the misuses of it? [00:00:39] So we are looking at all the horribles or potential horribles to stop that from happening, stopping scammers, stopping fraudsters, stopping pedophiles using our technology, stopping nation states to misuse it for national security type risks or influence operations to meddle with elections and things like that. [00:01:02] So we are really looking at the real life use cases. That is at the very end of the model development lifecycle. But we are also in a way at the spearhead and upstream of new types of uses of our technology to figure out are they actually bad, is something weird going on here, are people using this as a number of new use cases. [00:01:23] You can see that all the time. Humans are incredibly creative. We can think through all kinds of hypotheticals, but when it comes to humans, you're always surprised in how they're using this type of technology. [00:01:34] And then we are the ones who have to alert folks and figure out what's going on. I'm not a scientist, as I already mentioned, but as you can see, our team is quite multidisciplinary. [00:01:43] We have former law enforcement, former folks who worked on chemical or biological weapons, and obviously prohibiting those from being proliferated. But we also have engineers, we have general analysts, geopolitical analysts, as you can imagine, psychologists, product managers. All in a tiny, small team. [00:02:04] Which, as a first lesson learned, has brought us quite far. Bringing those folks close together, engineers with analysts, is somewhat of a success story, at least for ourselves, in bringing AI to the workflows that we're using it for. [00:02:18] So that's the key here. We are of course, I'm speaking here today because we're using AI in a way that it really is focused on the applied side. [00:02:27] So we are not developing models, but we are developing systems on top of these models to make our job as good as possible, and as focused as humanly possible. [00:02:41] So this is really important; we are working on very weak signals to figure out what's going on in our platform. And you all have heard, we have hundreds of millions of users, already quite a stack of products to make sure that they are safe. [00:02:57] Now how does that look like in terms of the very basic workflow that we are following? We usually work with some kind of signals, to look at, okay, something might be happening here. [00:03:08] We're seeing a weird M-dash on Twitter, right? You've all probably grappled with that one, and figure out, okay, is somebody generating these weird misinforming tweets? [00:03:20] Or is there somebody behind here using our models to automate large-scale misinformation campaigns? And this is kind of like an entry point, for example. [00:03:28] Then we try to bring all the evidence together and figure out what's going on here, and put that together in a brief, in a report, to make sure that we inform our partners' teams across the company and the public. [00:03:38] Because we're publishing around these findings that we have as well, on a more or less quarterly basis. We've just published something two weeks ago on how China, for example, tried to use our models to influence public opinion on AI and data centers in the US. [00:03:56] So that's kind of the work that we're doing, and this is kind of like the baseline workflow across the team, really boiled down to the essentials. [00:04:05] And when you look at the investigations that we're doing, like I said, there are fragments that we have to bring together, bring context together, figure out what our confidence level is in the first place, figure out a risk taxonomy and a prioritization framework to really be able to look at this at scale, and of course, no surprise, AI is all over the place here. [00:04:29] Why is that important, and where do we use AI? In this workflow, there's some of the key use cases that we have across the board, and across many, many of these workflows that are out there. [00:04:40] First of all, of course, there's a human that decides what actually matters. We have product policies, usage policies that we apply to our products. That's kind of the general guideline, and then we also, of course, want to figure out what may be bad that we haven't yet figured out is actually bad. [00:04:58] So there's this novel harms aspect.

[00:04:58] So there's this novel harms aspect to it, emerging risks that always pop up.

[00:05:03] So there's the human ideation, threat modeling in place, scenario planning, forecasting that sets around the guidance in the first place. Then what we really want to do is finding the needle in the haystack.

[00:05:12] We want to prevent any type of violence to be facilitated with our product, for example. So we need to find that early, find that one needle in the haystack.

[00:05:16] You know this from terrorism prevention kind of frameworks. This is where AI is really good. We have methods like embeddings, of course, classifiers that run cheaply for this kind of work to make sure that we find the semantic similarities in that ginormous haystack.

[00:05:27] Now what we also want to do is then cluster and find the patterns between these things in our ginormous data sets. We want to figure out, here's a good use case, right?

[00:05:50] There's a lot of folks that use JTBT for romantic advice. Clearly, there are edge cases around this that are likely harmful, so harassing your partner, for example, is probably not something that we want to facilitate, but the gray zones in that space are often very hard to figure out.

[00:06:14] But here AI does help us, does help us find these edge cases. We're able to cluster them and find those patterns. And then, of course, we have either already our models themselves grade these results for us based on a taxonomy that often they come up with in the first place based on the guidance, and then run this against that to go deeper with a human, obviously, to take the final look and investigate or potentially monitor something that we feel could be a problem in the future but looks still pretty much okay or really figure out, okay, this is just a very benign use case.

[00:06:50] We can leave that go. Now, of course, the loop is always important. Evaluations help with that a lot. Can we make this loop really work from the very first beginning?

[00:07:01] Can we reframe our own inquiry? Can we test our own biases that investigators, of course, bring themselves with AI itself and then run a loop even ideally even better to a point where when we wake up, our investigators have a potentially already pre-drafted report in their inbox from agentic systems that we use to then rapidly speed up, not only speed up the investigations, but also find things they had never thought about in the first place.

[00:07:32] So really augmenting the quality and speed of the work that we're doing. Now, ideally, the output is really something that brings in it all. It brings in the evidence, the prevalence, the severity, but also the context and timing.

[00:07:48] Think about somebody trying to develop a new fraud and scam based on a new retirement scheme in a country, right? That's usually when we see fraud come up because people are unsure what's happening, so there's some uncertainty with people, they need facts, and that's the perfect moment for a scammer to penetrate that system and push people into a fraudulent situation.

[00:08:12] Often, that context is very important because then we see, okay, there is a new retirement scheme happening in that country or geography, so that will likely potentially lead to an increased demand for scam-related material, so we can target our models more on these types of questions, to then also give us an estimate on where does the risk actually sit.

[00:08:35] In terms of harm to real people in the world. So that is something that also, channeling my own engineers here, they have done quite a bit in terms of a learning curve.

[00:08:45] They, when they started with us about two, two and a half years ago, were thinking, how do we actually make sure that AI is delivering what it should be? And at the beginning, everybody remembers hallucinations being quite a big problem, and they had to figure out how do we make hallucinations better.

[00:09:04] Now the solution that we as a best practice have now is not necessarily to correct the hallucination rates of models, because just six months later, they will be better at this. They will solve it.

[00:09:15] The increasing capabilities of the models will happen. But build for a system that can actually take care of these increased capabilities and not fix them in the first place.

[00:09:25] Because that's kind of a thing that you're working three months on fixing hallucination rates in a certain workflow, a new model comes out, it already does that for yourself. So kind of waste of time.

[00:09:35] So what do you do instead? You build a system that shows you the confidence interval or the confidence of the model itself in a certain result.

[00:09:42] So you build a system that's able to absorb the new capabilities that the model will have in two months, right? That's the constant. It's a similar challenge, but also an opportunity for us to do work very differently, not to build that very deterministic system.

[00:09:56] that very deterministic system but allow a system that goes with the increasing capabilities that these models will have next month basically.

[00:10:08] And what is really important what I already mentioned is, and these are some numbers in terms of how many influence operations we have stopped and publicly talked about, is really about the loop here.

[00:10:17] And the loop is important because at any point in this stage we can use AI to make it better and learn from it. So basically what you already have, and I talk about this a bit in a minute as well, if you let, let's say, a codex or another coding harness follow you in that loop, it can self-identify ways already to deploy agents to help you at every step.

[00:10:39] For example, orchestrate a team of experts, of agents that can help you find the right signals, investigate and use the context that you need, and then already route it to the right people if you needed to get fast to someone to respond quickly.

[00:11:00] And of course building a report, you all know, is something that's been around quite some time, but again this learning loop here is something that we particularly in the last quarter have focused quite a lot on.

[00:11:13] And very important to say is that we test and experiment a lot. What that means, you also need to give time for that. Like we all come from traditional tech or law enforcement spaces, where that wasn't necessarily normal, right?

[00:11:23] To sit back, do a hackathon, particularly with non-technical analysts and folks as well, to really figure out at what point in the workflows can we do something differently, figure out if it's working or not yet, and build it while we're actually flying the plane.

[00:11:37] Of course, you still need to make sure that you still deliver on your mission. But making time and space for folks to experiment, do a hackathon for non-engineers, really has driven quite some interesting results for folks that haven't come from the engineering background or even a research background.

[00:12:02] Now, and this was a fun one because Jason asked me, so how many agents are you using in your workflows?

[00:12:09] Now, this is a really tricky question and had me think quite a lot because there's one thing that we of course have central systems that do detection investigative work with tons of agents themselves, with swarm-based approaches that really go deep in large data sets to make sure that we find those needles, but also these edge cases that we want to see.

[00:12:36] But there is something happening, and we call this the codex moment, where every individual on the team will deploy agents for themselves.

[00:12:44] And that's as mundane as measuring their schedule to really copying and figuring out and being an assistant for each of the workflows they are running every week.

[00:12:53] And that means each individual may already have dozens of agents doing a loop for them every day. So that's one thing because that is really hard to figure out to just count them all.

[00:13:05] But then there's of course in those systematic areas, we have agents that are much more sophisticated and much more centralized. Think about red teaming, right, an area where we, in our area, tried to preempt misuses before they actually happen.

[00:13:20] So we can create personas at scale with characters and backgrounds and context that are entirely fake, but may represent a certain population we are concerned about and then deploy them at scale against our models to see what happens if you have a persona like that.

[00:13:35] We can do that at the thousands, right, or even millions. That itself, of course, is an agentic system.

[00:13:50] We have discovery tools that run around the clock to screen and retrieve, cluster and monitor. Usually when it comes at the filter stage, after the classifier has picked up certain signals, you need a greater, you need an agentic system that ideally have diverse expertise to challenge each other, judge.

[00:14:08] We just had the judge layer described by Gabriel as well. And then of course, really trying to build that AI investigator that can go out there and pre-draft the report.

[00:14:19] That is not just one agent, of course, that is an agentic system, because we can have them replicate certain roles. We have a director, you have those that look into the forensic evidence, in the external evidence, the editor, the verifier.

[00:14:43] And Codex can build this for you basically automatically already, which is incredibly cool and fascinating.

[00:14:49] And then of course, the report building, I think is the most obvious here, where agents come in as well.

[00:14:54] So the thousands here is a very directional indicator. It's definitely increasing every day,

[00:15:02] and I'm starting to burn 2 billion tokens with identics systems a day. So I even myself have to check in if I maybe have to reduce some of the agents.

[00:15:15] Now this is a bit of the, how do we bring these identity systems together in a bigger workflow. Here you can see, I think the key message here is to save costs, but also make it faster.

[00:15:28] You use usually at the first level, very broad level, you use classifiers, of course, because they are less expensive and can run on large data sets. You also use embedding systems to find those semantic similarities.

[00:15:41] And here you have how the agent and surface work together with certain types of analysts and profiles. And then when you want the really, really dedicated intelligent thought output, you use the higher reasoning models for graders, report building, and then of course, monitoring evaluations, which they can also even build themselves right now as well.

[00:16:02] The latest capabilities can run bespoke evals on your workflow without having to be an expert in evaluations. To then bring it all back, this feedback loop comes back on each slide, as you can tell, to improve the workflow throughout and make the graders, particularly graders, incredibly important, and their quality makes the difference in being able to filter to what actually matters, and reduce the noise from what you actually wanna find.

[00:16:32] Now, the big point here that I wanna take away, I want you to take away from our presentation here is that this is really not about software engineering. Obviously, we talk about science quite a bit already, but it's about, particularly I think, the opportunity lies in the non-software engineering type skill workflows.

[00:16:56] Because what we have is a situation where very different skillsets can now augment themselves with software tools bespoke to them and their needs. This is in a moment where we suddenly unlock capabilities that we have never seen before, right?

[00:17:14] In the investigative work, we always have the challenge that as soon as you bring out a report it's old, right? That's been always the case because something new happens either externally or internally that as soon as you share a report, it's already outdated the next minute.

[00:17:30] But you can build a system that can actually auto-update these reports, right? Agents run through incredibly new systems to update these reports on the spot.

[00:17:41] We have psychologists that can suddenly run their own evals because the models can help them build those for them. We have, of course, the quantitative analysts that can now be also data scientists that can be, that don't necessarily have to be able to code to build systems that are incredibly important for monitoring research and review.

[00:18:05] So this is something that we have found out is incredibly powerful. And almost everyone on my team has built some system to augment their own work, either to have information retrieval happening for them dedicatedly by running through external information to build reports automatically for themselves.

[00:18:26] And of course, the menial office task is something that everybody uses it for. But what is incredibly good, of course, to save time overall.

[00:18:37] And this is already my last slide. I think what is important is that the models themselves are like an engine for a car, right? They are incredibly powerful and are becoming more than just the engine because they can already natively use tools quite a bit, but you still need to build the system and figure out where our accountability sits, who is the driver, and usually that's the human, unless you get into self-driving spaces, right?

[00:19:04] And that's similar to how we're thinking about our own systems. Can we rely on the self-parking systems or can we already go fully self-driving autonomously? That always depends a bit on the space that we are in.

[00:19:18] Of course, our work is incredibly important. If we have to work with law enforcement, you need to be sure that this case is right, right? You probably definitely want a human in the loop here, but there are cases in fraud or scams, for example, where you can run things at scale and be much more automated.

[00:19:37] It also helps you, particularly in our space, to be incredibly privacy sensitive. We're working in a space where we need to protect the data of our users at all costs, so it is incredibly important that we don't look at everybody's chat history, obviously, and have these automated.

[00:19:52] obviously, and have these automated systems really only flag the very, very, very small amount of users that may misuse our tools in the first place. Thank you very much.