All Content

Given the computational cost and technical expertise required to train machine learning models, users may delegate the task of learning to a service provider. We show how a malicious learner can plant an undetectable backdoor into a classifier. On the surface, such a backdoored classifier behaves normally, but in reality, the learner maintains a mechanism for changing the classification of any input, with only a slight perturbation. Importantly, without the appropriate "backdoor key", the mechanism is hidden and cannot be detected by any computationally-bounded observer. We demonstrate two frameworks for planting undetectable backdoors, with incomparable guarantees.  Our construction of undetectable backdoors also sheds light on the related issue of robustness to adversarial examples. In particular, our construction can produce a classifier that is indistinguishable from an "adversarially robust" classifier, but where every input has an adversarial. Can backdoors be mitigated even if not detectable? Shafi will discuss a few approaches toward mitigation. This talk is based largely on collaborations with Kim, Shafer, Neekon, Vaikuntanathan, and Zamir.

Yonadav presents his research Practices for Governing Agentic AI Systems.

As AI gets more advanced and widely used, it is essential to involve the public in deciding how AI should behave in order to better align our models to the values of humanity. Last May, we announced the Democratic Inputs to AI grant program. We partnered with 10 teams out of nearly 1000 applicants to design, build, and test ideas that use democratic methods to decide the rules that govern AI systems. Throughout, the teams tackled challenges like recruiting diverse participants across the digital divide, producing a coherent output that represents diverse viewpoints, and designing processes with sufficient transparency to be trusted by the public. At OpenAI, we’re building on this momentum by designing an end-to-end process for collecting inputs from external stakeholders and using those inputs to train and shape the behavior of our models. Our goal is to design systems that incorporate public inputs to steer powerful AI models while addressing the above challenges. To help ensure that we continue to make progress on this research, we have formed a “Collective Alignment” team.

Collin Burns and Pavel Izmailov present their research, Weak-to-Strong Generalization

In this talk, Meissler shared his philosophy for integrating AI into all facets of life. He highlighted a framework built for leveraging custom prompts as APIs, as well as demonstrated several specific use cases that the speaker hopes will resonate with OpenAI Forum members and translate across disciplines and professional domains.

Earlier this year, Sam Altman, CEO and Co-Founder of OpenAI and David Kirtley, CEO and Founder of Helion convened at the OpenAI office among a small group of OpenAI Forum members to discuss the future of energy. This is the recording of their discussion.

Watch the demos presented by the recipients of OpenAI’s Democratic Inputs to AI Grant Program https://openai.com/blog/democratic-inputs-to-ai, who shared their ideas and processes with grant advisors, OpenAI team members, and the external AI research community (e.g., members of the Frontier Model Forum https://openai.com/blog/frontier-model-forum).

About the Talk: Worldcoin is creating a new identity and financial network that distinguishes humans from AI. This recording captured the the final in-person OpenAI Forum event of 2023 featuring Alex Blania, the CEO and co-founder of Tools for Humanity and Worldcoin, and Miles Brundage, head of policy research at OpenAI. Attendees learned about the Worldcoin systems designed to generate proof of personhood and ensure democratic access and governance of these systems in order to fairly distribute their benefits. We even featured the Worldcoin Orbs on site!  About the Speakers: Alex Blania is the CEO and Co-Founder of Tools for Humanity, the technology company building tools for the Worldcoin project. He is a Co-Founder of the Worldcoin protocol with Sam Altman and Max Novendstern. Alex is responsible for leading strategy, development, and execution of the technology and tools enabling Worldcoin to ensure everyone benefits fairly from the opportunities AI presents. Alex holds degrees in Industrial Engineering and Physics from the University of Erlangen-Nuremberg in Germany and studied physics at Caltech before dedicating his full time and attention to supporting Worldcoin. Miles Brundage is a researcher and research manager, passionate about the responsible governance of artificial intelligence. In 2018, he joined OpenAI, where he began as a Research Scientist and recently became Head of Policy Research. Before joining OpenAI, he was a Research Fellow at the University of Oxford's Future of Humanity Institute. He is currently an Affiliate of the Centre for the Governance of AI and a member of the AI Task Force at the Center for a New American Security. From 2018 through 2022, he served as a member of Axon's AI and Policing Technology Ethics Board. He completed a PhD in Human and Social Dimensions of Science and Technology from Arizona State University in 2019. Prior to graduate school, he worked at the Advanced Research Projects Agency - Energy (ARPA-E). His academic research has been supported by the National Science Foundation, the Bipartisan Policy Center, and the Future of Life Institute.