AI in Science & Research

OpenAI and nine national labs bring together leading scientists for first-of-its kind event.

About the Talk: Language models (LMs) are increasingly being used in open-ended contexts, where the opinions reflected by LMs in response to subjective queries can have a profound impact, both on user satisfaction, as well as shaping the views of society at large. In this work, we put forth a quantitative framework to investigate the opinions reflected by LMs -- by leveraging high-quality public opinion polls and their associated human responses. Using this framework, we create OpinionsQA, a new dataset for evaluating the alignment of LM opinions with those of 60 US demographic groups over topics ranging from abortion to automation. Across topics, we find substantial misalignment between the views reflected by current LMs and those of US demographic groups: on par with the Democrat-Republican divide on climate change. Notably, this misalignment persists even after explicitly steering the LMs towards particular demographic groups. Our analysis not only confirms prior observations about the left-leaning tendencies of some human feedback-tuned LMs, but also surfaces groups whose opinions are poorly reflected by current LMs (e.g., 65+ and widowed individuals).

The Importance of Science Communicator & Policy Research Roles with Miles Brundage, Head of Policy Research at OpenAI and Andrew Mayne, Science Communicator at OpenAI

About the Talk: Much of the value of labor in industrialized economies derives from the scarcity of expertise rather than from the scarcity of labor per se. In economic parlance, expertise denotes a specific body of knowledge or competency required for accomplishing a particular objective. Human expertise commands a market premium to the degree that it is, first, necessary for accomplishing valuable objectives, and second, scarce, meaning not possessed by most people. Will  AI increase the value of expertise by broadening its relevance and applicability? Or will it instead commodify expertise and undermine pay, even if jobs are not lost in net. Autor will present a simple framework for interpreting the relationship between technological change and expertise across three different technological revolutions. He will argue that, due to AI’s malleability and broad applicability, its labor market consequences will depend fundamentally on how firms, governments, NGOs, and universities (among others) invest to develop its capabilities and shape its applications.  

About the Talk: In recent years, large language models have greatly improved in their ability to perform complex multi-step reasoning. However, even state-of-the-art models still regularly produce logical mistakes. To train more reliable models, we can turn either to outcome supervision, which provides feedback for a final result, or process supervision, which provides feedback for each intermediate reasoning step. Given the importance of training reliable models, and given the high cost of human feedback, it is important to carefully compare the both methods. Recent work has already begun this comparison, but many questions still remain. We conduct our own investigation, finding that process supervision significantly outperforms outcome supervision for training models to solve problems from the challenging MATH dataset. Our process-supervised model solves 78% of problems from a representative subset of the MATH test set. Additionally, we show that active learning significantly improves the efficacy of process supervision. To support related research, we also release PRM800K, the complete dataset of 800,000 step-level human feedback labels used to train our best reward model. Full list of Authors: Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, Karl Cobbe

About the Talk: AI will have significant, far-reaching economic and societal impacts. Technology shapes the lives of individuals, how we interact with one another, and how society as a whole evolves. We believe that decisions about how AI systems behave should be shaped by diverse perspectives reflecting the public interest. Join Lama Ahmad (Policy Researcher at OpenAI) and Saffron Huang and Divya Siddarth (Co-Directors of the Collective Intelligence Project) in conversation to reflect on why public input matters for designing AI systems, and how these methods might be operationalized in practice. The Collective Intelligence Project White Paper: The Collective Intelligence Project (CIP) is an incubator for new governance models for transformative technology. CIP will focus on the research and development of collective intelligence capabilities: decision-making technologies, processes, and institutions that expand a group’s capacity to construct and cooperate towards shared goals. We will apply these capabilities to transformative technology: technological advances with a high likelihood of significantly altering our society. Read More About the OpenAI Grant, Democratic Inputs to AI

The Data Science for Social Justice Workshop (DSSJ), organized in partnership between UC Berkeley’s Graduate Division and D-Lab, is an 8-week program aiming to provide an introduction to data science for graduate students, grounded in critical approaches of data feminism, data activism, ethics, and critical race theory. Attendees receive training in natural language processing and leverage their skills to conduct discourse analysis on social media data in an interdisciplinary project. This workshop, about to conclude its third year, has trained over 75 graduate students across 20 disciplines. These students form a community of interdisciplinary scholar-activists who uphold a values-driven approach to data science and machine learning. In this event, Claudia von Vacano, Ph.D., Executive Director of D-Lab, introduces the Data Science for Social Justice Workshop, highlighting its goals, structure, and outcomes. Then, three students who have participated in the workshop – with diverse and rich personal and academic backgrounds – present lightning talks on their experience with DSSJ, highlighting their personal journeys, the projects they worked on, and what they gained from the workshop. The event will conclude with a Q&A and discussion on how workshops like DSSJ present novel opportunities to train a generation of interdisciplinary, diverse data-driven scientists who center values and social justice at the forefront of their work.

Informative session about OpenAI's Research Residency program, perfect for anyone interested in forging a career in AI, but without extensive experience in the domain. Our 6-month residency helps technical researchers from diverse fields transition into AI. Led by the program manager, Jackie Hehir, this session offers insights into the program's structure, benefits, and application process.The residency is an excellent way for people who are curious, passionate, and skilled to sharpen their focus on AI and machine learning and contribute to OpenAI’s mission of building AGI that benefits all of humanity. Learn more about the residency program and discover research blogs published by residents at the bottom of this page here.

Using the imaging of black holes as a case study, this talk highlights the key requirements for AI to make meaningful contributions to astrophysical research. Dr. Chan introduces several pioneering projects that are integrating AI into astrophysics, covering aspects such as instrumentation, simulations, data processing, and causal inference. He also discusses an innovative project aimed at enabling AI to gain scientific insights independently.

Hear from research leadership first hand about the significance of expert trainer contributions to the OpenAI mission.

The session featured several nonprofit organizations that utilize AI to drive social impact, emphasizing their long-standing involvement with the community. The discussion was facilitated by Nathan Chappell, a notable figure in AI fundraising, and included insights from a diverse group of panelists: Dupe Ajayi, Jodi Britton, Allison Fine, Anne Murphy, Gayle Roberts, Scott Rosenkrans, and Woodrow Rosenbaum. Each speaker shared their experiences and perspectives on integrating AI into their operations, illustrating AI's transformative potential in various sectors. The event highlighted the importance of AI in amplifying the efficiency and reach of nonprofit initiatives, suggesting a significant role for AI in addressing global challenges. The conversation also touched on the ethical considerations and the need for responsible AI use, ensuring that technological advancements align with human values and contribute positively to society. This gathering not only served as a platform for sharing knowledge and experiences but also fostered networking among community members with similar interests in AI applications. The dialogue underscored the critical role of AI in future developments across fields, advocating for continued exploration and adoption of AI technologies to enhance organizational impact and effectiveness.

Given the computational cost and technical expertise required to train machine learning models, users may delegate the task of learning to a service provider. We show how a malicious learner can plant an undetectable backdoor into a classifier. On the surface, such a backdoored classifier behaves normally, but in reality, the learner maintains a mechanism for changing the classification of any input, with only a slight perturbation. Importantly, without the appropriate "backdoor key", the mechanism is hidden and cannot be detected by any computationally-bounded observer. We demonstrate two frameworks for planting undetectable backdoors, with incomparable guarantees.  Our construction of undetectable backdoors also sheds light on the related issue of robustness to adversarial examples. In particular, our construction can produce a classifier that is indistinguishable from an "adversarially robust" classifier, but where every input has an adversarial. Can backdoors be mitigated even if not detectable? Shafi will discuss a few approaches toward mitigation. This talk is based largely on collaborations with Kim, Shafer, Neekon, Vaikuntanathan, and Zamir.

Yonadav presents his research Practices for Governing Agentic AI Systems.

As AI gets more advanced and widely used, it is essential to involve the public in deciding how AI should behave in order to better align our models to the values of humanity. Last May, we announced the Democratic Inputs to AI grant program. We partnered with 10 teams out of nearly 1000 applicants to design, build, and test ideas that use democratic methods to decide the rules that govern AI systems. Throughout, the teams tackled challenges like recruiting diverse participants across the digital divide, producing a coherent output that represents diverse viewpoints, and designing processes with sufficient transparency to be trusted by the public. At OpenAI, we’re building on this momentum by designing an end-to-end process for collecting inputs from external stakeholders and using those inputs to train and shape the behavior of our models. Our goal is to design systems that incorporate public inputs to steer powerful AI models while addressing the above challenges. To help ensure that we continue to make progress on this research, we have formed a “Collective Alignment” team.

In a recent talk at OpenAI, Lama Ahmad shared insights into OpenAI’s Red Teaming efforts, which play a critical role in ensuring the safety and reliability of AI systems. Hosted by Natalie Cone, OpenAI Forum’s Community Manager, the session opened with an opportunity for audience members to participate in cybersecurity initiatives at OpenAI. The primary focus of the event was red teaming AI systems—a process for identifying risks and vulnerabilities in models to improve their robustness. Red teaming, as Ahmad explained, is derived from cybersecurity practices, but has evolved to fit the AI industry’s needs. At its core, it’s a structured process for probing AI systems to identify harmful outputs, infrastructural threats, and other risks that could emerge during normal or adversarial use. Red teaming not only tests systems under potential misuse, but also evaluates normal user interactions to identify unintentional failures or undesirable outcomes, such as inaccurate outputs. Ahmad, who leads OpenAI’s external assessments of AI system impacts, emphasized that these efforts are vital to building safer, more reliable systems. Ahmad provided a detailed history of how OpenAI’s red teaming efforts have grown in tandem with its product development. She described how, during her tenure at OpenAI, the launch of systems like DALL-E 3 and ChatGPT greatly expanded the accessibility of AI tools to the public, making red teaming more important than ever. The accessibility of these tools, she noted, increases their impact across various domains, both positively and negatively, making it critical to assess the risks AI might pose to different groups of users. Ahmad outlined several key lessons learned from red teaming at OpenAI. First, red teaming is a “full stack policy challenge,” requiring coordination across different teams and expertise areas. It is not a one-time process, but must be continually integrated into the AI development lifecycle. Additionally, diverse perspectives are essential for understanding potential failure modes. Ahmad noted that OpenAI relies on internal teams, external experts, and automated systems to probe for risks. Automated red teaming, where models are used to generate test cases, is increasingly useful, but human experts remain crucial for understanding nuanced risks that automated methods might miss. Ahmad also highlighted specific examples from red teaming, such as the discovery of visual synonyms, where users can bypass content restrictions by using alternative terms. She pointed out how features like DALL-E’s inpainting tool, which allows users to edit parts of images, pose unique challenges that require both qualitative and quantitative risk assessments. Red teaming’s findings often lead to model-level mitigations, system-level safeguards like keyword blocklists, or even policy development to ensure safe and ethical use of AI systems. During the Q&A session, attendees raised questions about the challenges of red teaming in industries like life sciences and healthcare, where sensitive topics could lead to overly cautious models. Ahmad emphasized that red teaming is a measurement tool meant to track risks over time and is not designed to provide definitive solutions. Other audience members inquired about the risks of misinformation in AI systems, especially around elections. Ahmad assured participants that OpenAI is actively working to address these concerns, with red teaming efforts focused on areas like misinformation and bias. In conclusion, Ahmad stressed that as AI systems become more complex, red teaming will continue to evolve, combining human evaluations with automated testing to scale risk assessments. OpenAI’s iterative deployment model, she said, allows the company to learn from real-world use cases, ensuring that its systems are continuously improved. Although automated evaluations are valuable, human involvement remains crucial for addressing novel risks and building safer, more reliable AI systems.

Collin Burns and Pavel Izmailov present their research, Weak-to-Strong Generalization

In our conversation, we explored the fundamental principles of organization and function of biological neural networks. Anton Maximov provided an overview of imaging studies that have revealed the remarkable diversity of neurons in the brain and the complexity of their connections. His presentation began with the pioneering work of Santiago Ramón y Cajal and extended to contemporary research that integrates advanced imaging technologies with artificial intelligence. He discussed discoveries from his laboratory at Scripps, unveiling surprising new mechanisms by which neural circuits in the brain are reorganized during memory encoding. His presentation was engaging, with vibrant videos and images to showcase his findings.

Earlier this year, Sam Altman, CEO and Co-Founder of OpenAI and David Kirtley, CEO and Founder of Helion convened at the OpenAI office among a small group of OpenAI Forum members to discuss the future of energy. This is the recording of their discussion.

Watch the demos presented by the recipients of OpenAI’s Democratic Inputs to AI Grant Program https://openai.com/blog/democratic-inputs-to-ai, who shared their ideas and processes with grant advisors, OpenAI team members, and the external AI research community (e.g., members of the Frontier Model Forum https://openai.com/blog/frontier-model-forum).

The presentation from Isa Fulford and Edward Sun offers an in-depth look into “Deep Research,” a capability within ChatGPT powered by a fine-tuned version of the o3 model. The model is built with agentic capabilities that enable it to autonomously conduct complex, long-horizon research tasks involving browsing, reasoning, data processing, and synthesis. Deep Research is positioned as a leap toward more capable AI agents that save users significant time and deliver high-quality, sourced outputs. The presentation also showcases how reinforcement learning, reasoning models, and safety measures contribute to creating a robust system meant to support real-world professional tasks—particularly in business, science, medicine, and academia.

During the virtual event on December 3rd, Prof. Terence Tao and OpenAI's Mark Chen and James Donovan engaged in a deep discussion on the intersection of AI and mathematics. They explored how AI models, particularly new reasoning models, could enhance traditional mathematical problem-solving and potentially transform mathematical research. The speakers discussed the integration of AI into various scientific fields, emphasizing AI's role in accelerating discovery and innovation. Key topics included the challenges of AI in understanding and contributing to complex mathematical proofs, the evolving nature of mathematical research with AI integration, and the future of collaboration between AI and human mathematicians. The conversation highlighted both the potential and the current limitations of AI in advancing mathematical sciences.

This presentation showcased how Dr. Shahram Yazdani and his interdisciplinary team at UCLA are using AI and OpenAI’s language models to enhance diagnostic precision and address systemic inequities in healthcare. By leveraging text embeddings and semantic vectorization, their projects aim to reduce diagnostic errors, predict hospitalizations, and personalize care—especially for children with complex medical needs in underserved communities.

In this Forum session, OpenAI’s VP of Science Kevin Weil and Brian Spears, Director of Lawrence Livermore National Laboratory’s AI Innovation Incubator (AI3), will explore how advanced AI systems are beginning to make direct, measurable contributions to scientific research. The discussion will highlight the OpenAI–LLNL partnership and what it looks like when frontier reasoning models are embedded in real scientific workflows—from accelerating hypothesis generation and analyzing complex datasets to uncovering connections that were previously out of reach. Weil will share the vision behind OpenAI for Science, including the ambition to “compress 25 years of scientific progress into 5,” by giving researchers powerful new instruments for discovery. Spears will offer the lab-level perspective on how AI is already expanding the pace, scale, and ambition of work across fields like energy, materials science, and high-performance computing. By bringing frontier AI into some of the nation’s most capable—and most secure—research institutions, OpenAI and the national labs are working together to build a more rapid, reliable, and resilient model for turning scientific insight into real-world impact.

At a recent OpenAI Forum conversation, GitLab co-founder and Executive Chair Sid Sijbrandij joined geneticist Jacob Stern to discuss how they have used AI, advanced diagnostics, and personalized treatment design in response to Sid’s osteosarcoma diagnosis. Hosted by Chris Nicholson and introduced by OpenAI researcher Scott McKinney, the session focused on what becomes possible when patients, researchers, and technologists work together to go beyond standard care, especially in the context of a rare and aggressive cancer. Sid and Jacob described building a highly individualized approach after standard options became limited, combining extensive diagnostics with AI-assisted analysis to better understand Sid’s specific tumor biology. Their work has included single-cell sequencing, DNA and RNA sequencing, targeted imaging, organoid testing, and the development of experimental treatment strategies such as a personalized mRNA vaccine and engineered cell therapies. Jacob explained that AI helped accelerate literature review, hypothesis generation, and bioinformatics analysis, allowing him to collaborate more effectively with specialists and move faster in areas where time and precision were critical. A central message of the event was that AI can help make medicine more personalized, iterative, and accessible over time. Rather than presenting AI as a replacement for doctors or researchers, the speakers emphasized its value as a tool for helping patients and experts interpret complex data, explore new options, and ask better questions. The conversation closed on a hopeful note: Sid shared that after targeted radioactive treatment and surgery, there is currently no evidence of disease, and both speakers underscored their broader goal of helping make these kinds of patient-centered approaches easier for others in the future.

This conversation framed biology as a field moving from description to prediction. Grace Zheng emphasized that modern sequencing, imaging, single-cell measurement, and editing tools are making it possible to see biological systems more realistically and model how changes may affect outcomes. Xin Jin described predictive biology as a shift from asking what something is to asking what happens if it changes, whether through a mutation or a drug intervention. Natalie Cone connected that framing to the broader OpenAI science effort, including GPT-Rosalind, which is intended to uplift life science research and accelerate discovery. The discussion repeatedly returned to the idea that biology is too complex for any one lab to measure experimentally in full, which is why AI-assisted prediction can meaningfully change research and medicine. While we didn’t have time to get through all of the audience questions live, Grace and Xin kindly followed up with written responses linked here. https://tinyurl.com/44jzepa9

For families living with rare disease, the search for a diagnosis can stretch across years; this conversation made that uncertainty personal through Stav Rones’s story of finally finding a name for his condition. Researchers from Boston Children’s Hospital, Harvard, and OpenAI described how an AI-assisted workflow helped sift through genetic data and scientific literature, contributing to 18 diagnoses across 376 difficult cases. Just as importantly, they were clear that AI is not replacing clinicians: it narrows the field, presents the evidence, and leaves experts to test, interpret, and ultimately return answers to patients. The event offered a hopeful glimpse of a future in which more families can move sooner from uncertainty to understanding, community, and care.