OpenAI Residency Program Info Session

Informative session about OpenAI's Research Residency program, perfect for anyone interested in forging a career in AI, but without extensive experience in the domain. Our 6-month residency helps technical researchers from diverse fields transition into AI.

Led by the program manager, Jackie Hehir, this session offers insights into the program's structure, benefits, and application process.The residency is an excellent way for people who are curious, passionate, and skilled to sharpen their focus on AI and machine learning and contribute to OpenAI’s mission of building AGI that benefits all of humanity. Learn more about the residency program and discover research blogs published by residents at the bottom of this page here.

Hi, everyone. Welcome. As you might notice, if you've been with us for a while, the format of this event is a little different than what we're used to. Tonight, we're live streaming this event in order to accommodate a large audience registration. But don't worry, we'll also continue to offer the more intimate meeting style events in the future. And also the in-person events at OpenAI will continue to be streaming to a virtual audience.

I'd like to share tonight before we get started that I'm looking for some technical members of the forum who'd like to be paired with less technical members in the community for AI consultation and technical enablement. We have nonprofits in the forum and people from higher education with awesome ideas about how to integrate AI into their programs and socially beneficial projects, but could use some technical support from you. If this sounds interesting, and if you have the capacity to donate some of your time, please DM me in the wake of this event.

Let's get this party started. I'm Natalie Cone, your OpenAI forum community manager. I like to start our talks by reminding us of OpenAI's mission, which is to ensure that artificial general intelligence, AGI, by which we mean highly autonomous systems that outperform humans at most economically valuable work, benefits all of humanity.

Welcome to the OpenAI residency program info session. I hope by now you all feel that we're listening to you, your questions, comments, concerns are being addressed. And we hosted Bryden previously, who talked a little bit about alternative paths to entering AI, specifically OpenAI, and there were a lot of questions that surfaced very specifically about the research residency. And that's why we decided to host this event this evening.

So we'll start the evening by hearing from Jackie Hehir, OpenAI's research residency program manager, following by Hart Andrin, who will share his experience transitioning from the research residency program to a full time role at OpenAI. After the presentations, you'll have the opportunity to ask both speakers questions. Because tonight's event, the format is a little bit different, we ask that you drop your questions in the Q&A field on the right of the screen, and I will ask them for you to our speakers tonight.

Before we close out the night, there'll be an optional networking session to connect with other attendees. And you'll be able to find that in the left of your screen in the one on one match tab.

But before I pass the mic to Jackie and Hart, I'd like to share a little bit about our speakers with you all.

Jackie joined OpenAI almost three years ago. Before her time at OpenAI, she served as head of program management at Amplitude and Intercom. Jacqueline holds a Bachelor of Arts in Government and French from Harvard University where she was a varsity lacrosse player as well as a Master of Public Health in Health Policy and Management from Columbia University.

Hart Andrin went through the residency program last year and has been a research scientist at OpenAI for the past 15 months, working on the Frontiers team. Prior to this, Hart was a quantitative researcher for several years at various various hedge funds, working on applying statistical techniques to the stock market. Hart has a formal background in mathematics and his non traditional background and diverse perspective have helped guide research efforts at OpenAI in unexpected and impactful directions.

Without further ado, I'd like to welcome Jackie to the screen.

Hi, everyone.

Hey, thanks for having me. I'm happy to be here.

My pleasure. I know that the community is really excited to hear from you. So thank you for your time. And I'm going to pass the mic over to you and I'll come back later for the Q&A.

Cool. Well, as Natalie so politely introduced me, my name is Jackie Hehir, and I do have the privilege of running the residency program. It's not something that I had originally anticipated would be part of my life. And it's been the most happy coincidence that I stumbled into.

I want to start with a little bit of history about the residency program. And then I'm going to spend a lot of time, hopefully answering some of your questions a little bit proactively around what the residency program is, what we look for in an ideal resident, how the interview process runs, and ultimately what we hope that both the resident and OpenAI gain from the experience.

First and foremost, I want to start by just acknowledging that OpenAI is long home to a lot of self-taught and self-trained people. And I think that the residency program really embodies the spirit of that. We are thoughtful in thinking about how we grow our team organically, and that we really want to stave off an echo chamber of only recruiting and bringing people into our research fold that are classically trained in machine learning and artificial intelligence.

So thus was born the residency program, and it actually launched over two years ago in 2022. But prior to that, we had been running something called Fellows and Scholars. And in 2022, we actually merged the two programs together, sort of looking at the best of both worlds. The Fellows program was looking at people who were in adjacent STEM fields, and the Scholars program was looking at people who were actively pursuing PhDs or higher degree programs. And what we found is that actually trying to build a diverse research organization probably should pull down from both of those bodies.

So what we think about the program today and its current origin is that we're continuing to look for people from outside of the domain of artificial intelligence and machine learning who have a real aptitude and interest in pursuing careers in this space, but don't really know how to get their foothold into it. They are not really interested or seeking out a PhD in machine learning or AI. They're not working at other artificial intelligence labs, and they're really passionate about the space.

The way that we've constructed it, and we've been running this program, as I mentioned, for several years, we have a lot of data points. And like the good scientists that we are, we love a good data set. And so now that we have been running it for so long, we've really been able to fine-tune, joke intended, what we look for in our residents and how to run the program super well.

But at its core, it is a six-month program. And when we talk about looking for researchers outside of the AI and ML domain, what we are really looking for are people in adjacent STEM fields. So think applied mathematics, physics, neuroscience, even biophysics, chemistry. We want to see people who have somewhat established careers in research, who have kind of the foundations or the building blocks of what it means to run a great research project.

What we end up doing is we match a resident to a specific team, and that team could fit anywhere within our basic research organization. Our research organization is big and multifaceted. We have teams that cover all sorts of things ranging from Sora and Dali to our latest models that serve Chachi BT. So there is a plethora of spaces for residents to be tied to, but ultimately the feel of the experience should be pretty similar regardless of what team you're mapped to.

The team would offer you two things. One, you would have a dedicated mentor. And unlike a full-time member of staff who might join from, say, another research organization or a PhD in AI and ML, the resident and the mentor really spend the first kind of 60 to 90 days working very closely together. And the intention of that is that the mentor is there to help guide you through your first couple of weeks of your initial project.

After that initial project is over, you as a resident will actually be assigned a very meaningful project. And this project work is top of the list. This is not an internship where you're sort of being tasked with maybe a side project or something that's not really top of mind or relevant. These are really mission-critical projects, but they're also designed to be really good learning experiences. So the ramp up while steep is intended to be something that you can naturally teach yourself and learn through the project work.

And the mentor is there to be a little bit of your coach, right? They're not going to be there day-to-day kind of giving you tasks and asking you to execute on them. You are asked to independently and autonomously work, but the mentor is there to be your friend and also your advocate to help you kind of figure out how to avoid those potholes, if you will, in your research project and not go down a rabbit hole.

We also have an awesome community of resident alumnus, including HART, and we have at any given moment anywhere from 10 to 20 residents. So there's a really great community as well for you to reach out and seek guidance. This is not supposed to be an island of one situation.

At the end of the residency, one of my favorite anecdotes is that when I ask a resident, what does it feel like when you finish your residency and now you're a full-time member of staff? And I think it really speaks to the actual experience that they said. It really doesn't feel any different. I really felt like I was contributing pretty quickly, but the biggest delta for them is that they're just a lot faster. They sort of share that at the beginning, they felt like they never worked so hard and they were working faster than they'd ever worked before. And then they look back in hindsight and think, wow, I'm just so much faster. And they attribute that to just the quality of people that they're working with.

The mentors, their managers, and their peers are really exceptional. We have a very, very talent dense and talent rich research base. And because we are home to these self-taught and self-trained people, residents are really embraced because everybody believes that there is a foundational component to becoming a great researcher.

but that it is a learnable skill. So what are the requirements? So the begrudge of a lot of people within the recruiting industry, I don't have an archetype of what a great resident looks like. And I actually don't have a ton of requirements.

In fact, when we post the role, it's very loose in terms of what we define of requirements. We actually don't have any formal education requirements or work requirements. You don't need to have worked at X number of places for a certain amount of time, but we do hold an extremely high technical bar. And honestly, the technical bar for a resident is at parity with a full-time member of staff. It's just a different bar.

The reason for that is that the foundations of becoming a great ML researcher, which are foundational in mathematics and coding, we need to have residents show up on day one with those two foundation pieces pretty well developed. So while you don't need to have a degree in advanced mathematics, you do need to be really comfortable with advanced math concepts. So think about linear algebra, statistics, and probability. Those are pretty basic components of any foundations in machine learning and artificial intelligence.

The other thing that perhaps takes people by surprise is that while this is a research residency program, our research roles are very engineering forward. And so we do need residents to be extremely proficient in programming languages. So something that I look for when I'm looking at residents is has this person ever prototyped something, done an open source project? Did they study computer science in their undergraduate? Were they a software engineer at a startup over a summer break? Something of that nature where you've worked within a shared code base with other engineers and you understand these components. It's not a hard requirement, but it is a really foundational skillset. And the interview process will test both the math and the coding, regardless again of what team you're placed on.

So what's nice about the residency is that we have established a baseline of what we look for, which is technical, but in terms of other components of your background, be it you study neuroscience, you're a classically trained musician, those are all great components that we also look for, but we absolutely need to see really strong math and programming.

How to apply. So we do post the job a few times a year, generally in March and October and occasionally another time in the summer. We are expecting to repost the role this October and we've adopted a rolling start date.

Hart mentioned that he is part of a cohort class. We have done cohort models in the past, but what we found to be true is that our research teams operate at the speed of light effectively. And so our hiring needs also need to pivot to adopt to their team's hiring needs.

And so having fixed start dates limited us both in terms of the candidates that we could recruit as well as team's needs. So when we do post the role externally, what we're looking for is basically people who can start within, I would say, anywhere from 60 to 100 days from that day of posting.

So if we post the role in October, but you're currently pursuing a degree or you're tied up on a project and you're really not available until late summer or end of next year, I would recommend holding off on applying because I'm going to want to ensure that your availability is pretty immediate after the application window.

We always post to LinkedIn and I would recommend following us. Sometimes we also tweet out when we post the requisite because it is a very popular application.

How to prepare. I would say the number one thing that would be helpful to see on a resume is either access to your GitHub or a personal website that sort of showcases a portfolio of your research work. What we're looking for, as I mentioned, are researchers who are somewhat established in another domain that want to transition.

So it's really helpful for the program team to see what your research is about now. And there's nothing quite like looking at a personal website of portfolios or me having the ability to look through some of your open source work is really important that I have a better understanding of who you are as a researcher today.

I also love looking at if you're recently published or any talks that you're giving, include those on your CV. As I mentioned, the coding component is really, really critical to becoming a researcher here at OpenAI. So if you've never done industry coding interviews before, I strongly urge you to practice your coding.

LeetCode, HackerRank, CodeSignal, they can all help. You can also reach out to friends who perhaps work in the industry. We really encourage you to feel like you're stepping into the interview process as well prepared as you can. And these interviews will be very rigorous and focused on coding.

Finally, I always suggest that you read OpenAI's research mission and vision. It's a really compelling company that we work for. And a lot of our behavioral interviews will circle back to the mission and vision.

Finally, I wanna just kind of recap some FAQs that I often get asked. One of my favorite questions is, is this just an internship with a new name?

And I really wanna underscore that this is not an internship. Residency is definitely a chance for people to transition from one role to another. It's not something that's supposed to be a continuing education of their existing career. And we also want residents ideally at the end of the six months to remain on as full-time members of staff. And so it's something that we really want people to take a lot of weight into of, is this the right time and moment in my life for me to perhaps leave this other passion to transition into AI and ML?

Also, this is important to note for anybody who perhaps holds an academic position. We don't allow simultaneous enrollment.

So while we have employed people who previously held maybe assistant professorships or were on a tenure track, you do need to take a formal leave of absence from any other position that you have. Simultaneous enrollment also applies to any students. I know that that is a challenge, particularly for people who are pursuing degrees, but the residency program is not going anywhere.

We are going to be a loud and proud group in 2025. So please pick the time that is most appropriate for you and just know that we're here.

Is there a lot of coding and programming? Yes, I cannot stress this enough and underscore it. Research roles are programming heavy. It's actually a core part of the responsibility of our researchers. It's also something that we look for people that really enjoy this work. You will be required to do a lot of this during the technical evaluations, but even when you're here at OpenAI, it is a core part of the teamwork.

I think people who genuinely have a passion for it really love it. And I don't want you to feel like we're misleading you that these research roles don't include it.

Another common question is with respect to location. Our residents are San Francisco based, and this is because this is the hub of our research teams. And this is a fully immersive experience. We definitely want the residents to be in office.

We follow a three day a week hybrid office environment. So Monday, Tuesday, Wednesday in office, Thursday and Friday are remote. What we look for here is that because the residents are transitioning their career or some other component of their life, we want you to feel like you have that community in person. We also offer relocation assistance and we can also support work authorization.

One of the cool things about being a resident at OpenAI is that you are treated like a full-time member of staff with respect to how we employ you. You're eligible for all of our health and wellness benefits, a 401k, transportation. It is a fully salaried position.

And so that's something else to keep in mind is that we try to make the six month experience as supported as possible. And then do all residents convert to full-time? And the answer is no.

But I wanna underscore the fact that we do have a very high rate of success. And we actually attribute that to two things. One, we do a robust interview process for every single resident. And we take a lot of time to think through where that resident would be most successful. So something that is part of my responsibility is understanding who you are as a researcher today and find analogous and comparable skillsets to other teams.

Within our research organization, there are a broad spectrum of different subspecializations. And as we've gotten bigger, some teams have become more specific. And certain teams will seek out candidates that have complementary skillsets.

And that's part of my role is to sort of deduce what roles would make most sense for this resident. We really want residents to be successful because we're investing a lot of time into you. Also, I like to call out that there is no competition between residents.

Every single resident that is hired has the same and equal chance of becoming successful full-time member of staff. There is no sort of vying for the end goal of there's multiple residents on a team and only one person will get a full-time offer.

Any resident that is hired is eligible to become a full-time member of staff.

So with that, I would love to welcome Hart to the discussion because I am sure that there are a lot of questions about what it's like to be a resident. And Hart can really speak to that.

So to kick us off, I'll ask a couple of questions to Hart just to warm us up. And then Natalie has been kind enough.

to help facilitate questions from the audience. Hey, guys. As Natalie and Jackie introduced, my name's Hart, and yeah, I was a resident last year and have been working at OpenAI for the past 15 months now. Previously studied mathematics and maybe took one or two courses in college on AI and machine learning, and then later transitioned to doing industry research in the form of quantitative research at various hedge funds, so doing honestly maybe pretty similar problems in a different domain space of regression style problems. But I do think that there are important distinctions between the two that I'm happy to talk about as well. But before getting too much into that, I'd love to pass it off to Jackie and hear some of the questions that I'm sure you get asked all the time.

Yeah. So thank you for providing context around what you were doing before, but would you be able to walk us through a little bit of how what you did before either prepared you for residency and how similar that work was to what you do now?

Yeah. So I think first and foremost, I want to echo what you said about the engineering skills. So my previous work as well was very much engineering focused, maybe a little bit less than currently now, but ultimately similar skill set in the sense that you have big systems and you do some various different regression style problems around ultimately we have some inputs, we want to predict some outputs. So from a technical perspective, actually the transition for me wasn't too, too bad on that front, but I do think that the experience that I have between the two different industries was very, very large and I attribute that to culture, mission, drive and collaboration. It's, I think it's the coolest place ever to work here in the sense that some people just have an exciting result one day and they're super excited and race over and tell a group of like whoever's happens to be sitting there and then everyone gets excited together. And I think that that's just a very cool part to be a cool community to be a part of. And I think that that's, you know, I hadn't really seen that before as well. And maybe the other point that I mentioned that I want to underscore is the collaboration aspect as well. I think really everyone here is just like on the same boat and that's so obvious. You have some projects that if, for example, someone can help answer and unblock you, they're just like everyone is kind of like all hands on deck. And I think that it really feels like part of a team in ways that research labs that I've been a part of in the past, maybe, you know, a little bit less of that focus.

Yeah. What do you attribute that to?

I mean, I just think that people here ultimately really do believe in the mission that what we are working on is very important. And at the end of the day, we all want to just have AI be built and distributed equitably for the benefit of humanity. And, you know, you can see it when people walk in every day that this is their goal and they will -- people here are doers. They want to get this done and they will, you know, of course, do whatever they can. And that obviously a lot of that is working together. Everyone here, you know, again, very smart, very excited to help each other. And then this kind of energy bounces and is an infectious energy is what I always say.

That's great. So, Hart, one thing that I failed to mention during my presentation was a little bit of the structure of the residency. I alluded to a bit, but residents are assigned meaningful projects during their six months. And there is a checkpoint naturally at the halfway point where you do a midterm presentation, and then ultimately it culminates in a final presentation. Well, I know you probably can't speak to the specifics of the project that you did. Could you describe for the community a little bit about what that experience was like of being a resident and having this massive project assigned to you with a mentor? What was that like?

Yeah, absolutely. So, first and foremost, I completely echo what you're saying in the sense that it really does feel like you're a full-time basically right away. I think people not on your team, certainly this was the case for me and I think is the case for most residents, actually just think you're full-time just on the basis of the responsibilities that you're given and the ownership that you have. So, yeah, so maybe my question would be what is a general, or my answer rather, what is a general project that one might work on over a six-month period? And so a project that my team actually recently published that I can kind of talk about getting a little bit into the technical details of, you know, we have, well, if we have GPT generate a sequence of steps that ultimately answers like a math problem, for example, you can build a discriminator that identifies, that is trained to say is this step, you know, correct or incorrect and so on and so forth. And then from this, you can generate a bunch of different rollouts and whichever rollout the discriminator likes the best, you might say we have more confidence in this than anything else. And it's actually recently published, if anyone listening, it's called Let's Verify Step-by-Step and maybe there'll be a link provided to that paper as well, but that would be like an example of something that I would have worked on. I personally didn't work on it, but this is an example that was published recently that I think is pretty representative. And so maybe to go in a little bit more into kind of the step-by-step, you might do like, you know, efforts in terms of data collection, efforts in terms of training the discriminator from this data, efforts in terms of, okay, well, these results don't necessarily look like what we were expecting or maybe they do, but like how do we iterate on this? And this all ties back to the scientific method that I'm sure we're all familiar with. And so just, yeah, giving you a sense of like, you know, of course the, you're responsible end-to-end for, again, the data collection, designing the hypothesis space, building the system to test the hypothesis, and then looking at the results and designing metrics to be able to analyze the results to say, how do we like improve on our V2 and V3s and so on.

That's amazing. I wanted to ask you, hypothetically, if you had not done the residency last year and you had not joined OpenAI, what do you think you would be doing now?

Okay, so I actually want to decompose this answer into two parts. I think I could answer this one way. If I didn't join knowing what I knew then, I probably would have continued being a quant or maybe going down math academia route just because this is something I really enjoy. I'm pretty good at it. And, you know, of course, why not kind of thing. But knowing what I know now, if I didn't join OpenAI, I mean, I kind of just think AI is the future, to be honest. And so I would say I think there's a lot of stuff that is actually currently underrepresented, though, in the AI space. So I think OpenAI obviously does an awesome job of frontiers research and delivering this research on products. But I think there's a lot of specializations that I'm personally really excited about in terms of, for example, education and health care. I know that there's a lot of focus on that. I don't think that there's enough personally. And then also just in terms of policy research, making sure that, you know, the delivery and distribution of this software is safe and equitable and all that fun stuff. I think that these are things that are really, really important as well and require like, you know, similar but not the same skill set that I think, you know, I'd be working on if potentially not here.

Cool. Do you think that there's something that really helped you be successful as a resident?

Yeah. So successful as a resident, I mean, it's so obvious to me now and it wasn't obvious to me when I joined, actually, was to just be really, really scrappy. I think here there's just a general sense of, you know, things move fast and that's great. You get good results quickly. Everyone's excited. But the other side of the coin of this is that things break pretty often in terms of like, oh, people merge software at the same time, software changes. And then as a result, how do you get unstuck? And I think it's really easy to get stuck and be blocked and be blocked for, you know, days, weeks. And I just think you want to keep up with the excitement. And so this kind of scrappiness of, you notice this blocker, how do I get unblocked as quickly as possible? Find the person who like, you know, maybe made this change, have them explain like exactly what's going on, unblock this yourself. LLMs are pretty good at this kind of stuff as well. But just find a way to, like, if you have a problem, either find a way to get unstuck or change the problem to something tractable. So that would be 100% number one. Number two, I really do think this job, this company is what you make of it more than what I've seen anywhere else in the sense that it's very easy to be head down, do your work and produce awesome results. And that's great. But at the same time, I think that there's a lot happening at this company. And I think it's really easy to not soak it all in, if that makes sense. So I would really encourage my past self and anyone who's joining in the future to really be proactive in terms of like all the other things that are being worked on by various teams. I would encourage coffee chats with anyone who seems to, I guess, entertain it working on something that you're interested in as well. I think it's very important to widen your horizons beyond just the exact work that you're working on, both from like, it's just so freaking cool to be surrounded by all these people doing cool things. And also just cross team collaboration, I think is building research ideas from these discussions happens all the time. And it's up to you to find those.

Okay, Hart, I have one final question for you, which is looking

back at your initial days as a resident to where you are now. What's the biggest change that you see in yourself? I mean, I think for myself, maybe I alluded to this before. There's just a sense of scrappiness, of I just want to get stuff done in a way that maybe I didn't see in myself the same way before. But again, this company and this culture has a way of bringing that out of you because you see other people doing something similar and you see how rewarding and satisfying it can be. And really seeing your ideas come to life, like when you have your research idea really beat baselines, that's so satisfying and it's addictive. You just want to keep going. And so, yeah, I mean, I think the biggest difference is generally the scrappiness, generally the not necessarily knowing everything in the moment, but acknowledging, OK, here's a foothold into this problem and we'll pursue it and then get data and figure out from there where to go next. So just a general sense of being comfortable with being uncomfortable while markedly still going forward in the right direction.

Cool. OK, I think those are all of my major questions. Natalie, I'd welcome you to join and let us know what the audience has to ask for us.

Team, I wish we would have done this a long time ago. This was actually really great information for me. And I really appreciated listening to both of you. That was an awesome presentation, Jacqueline. And Hart, I will echo that I had no idea you were a research resident. I passed you in the halls in San Francisco HQ and definitely just thought you were a full-time member of the Frontiers team from the very beginning. So that's true. I can tell from an outsider looking in, like not a non-AI research scientist, that you cannot distinguish the research residents from the full-time employees. So that's super rad.

We have lots of questions. I'm going to answer the first one because I think I can answer it, actually. And this question was reiterated a few times in a few different ways. But if you have a humanist background or a non-technical background, what are other ways you can get involved in open AI programs? And if you guys don't mind, I'll take that one. But this Open AI Forum community was originally built to actually bring together expertise from all corners of the world and different domains so that we can collaborate with you on all sorts of different types of research. So while the research residency does require a high level of technicality, we hire PhDs all the time to support expert data collection and creating expert evaluation model data sets for the models. So we work with linguistic PhDs. We work with people from social science backgrounds. And you get to collaborate really intimately with our research scientists in that way as well. And actually, Ben Kinsella, who's on the call right now, he's an Open AI Forum ambassador. And he has a PhD in linguistics. And he helps me build community in the humanities. And he advises. And he evaluates the model. So there's lots of opportunities to get involved, even if this is not the specific one for you.

So let's see. Do you have any research residents that involve 3D and or simulated worlds? And that is from Ari Kalinowski. So I would say anyone who's exploring, generally speaking, multimodality in any faculty, be that robotics or some component of maybe in bioengineering where you're looking at complex cell distribution, MRI imaging, all of that is kind of my recommendation whenever you think about these other sciences is try to distill it down to the foundational basics. And so if you're thinking about imaging, I'm immediately going towards a physics, some sort of component of understanding time and space. There's mathematics in there. And I see a lot of beautiful overlap. And so part of my job is to be a little bit off the walls in presenting candidates that have these disparate and different backgrounds. Anecdotally, we've hired people who are professional poker players because it turns out that they're really good at statistics and math. Were they researchers? No. But did they have great intuition? Could they understand the scientific method? Yes. And so that's kind of the awesome part about residency is that the team sort of knows that they're getting an incomplete person. And let's be honest, we're all incomplete. It's just that the team needs to be able to understand, how am I going to help Hart sort of complete that picture to become the great research scientist that he could already be? It's just getting that extra nudge. So my recommendation is to always think about the basics of what goes into your science research today and start from there.

Great answer, Jacqueline. Anything you want to contribute, Hart, or was that pretty?

Yeah. I mean, I think I'll just echo what Jackie was saying in terms of I do think that the problems being solved are quite dynamic. And I can actually say from when I joined to now, I think that the problem space has actually changed quite a bit. And I do think it's important to more focusing on the fundamentals, the foundations is important. Of course, you have different problems that you care about solving more than others. But I do also think wanting to have the highest impact and solving the task at hand, be very flexible. This is something that maybe also I didn't appreciate when I joined as well. But it's very clear that flexible and research direction, I think, is very important. Thanks, team.

Another question from Maxwell Hull. What programming languages are the most important to know to work as a resident? Python. I think that that's basically the default. There might be other teams that work on other programming languages. I personally only program in Python. Actually, everyone I work with only programs in Python. Yeah, pretty straightforward answer. Good to know.

From Michelle Chirichi. Given the expressed preference of, I answered this one. It was about humanists. We'll move on to the other one. Can you get feedback on the technical tests? I guess the technical interviews throughout the residency application. So we don't give direct feedback about each individual's performance on technical assessments. But what I like to share with people is that assessments are imperfect. Interviewing is imperfect. And these are moments in time. And so to remember that if you don't meet the technical bar today, just keep in mind, as I outlined before, the technical bar for resident is equal to that of a full time. We just chose a different bar. We're just thinking about you as someone looking at rock solid foundation as opposed to super advanced deep ML understanding. You will go through different technical interview loops than a full time member of staff. We thought if we put you through the same loop as a full time member of staff, we'd sort of be stacking the cards in favor of only bringing in people with ML and AI and sort of defeat the purpose of a program.

Awesome. Thank you, Jacqueline. This question is from my friend Mehul, actually. Is the residency program for machine learning engineers with limited research experience, is there a place for that? There is. I think one thing that we're seeing in research, and you can probably speak to this too in terms of the origin story of how our research team started, which was like 10 people in a room to now being a couple hundred people big. As you get bigger, you get more specific. And certain teams have different specificity needs. And so depending on where you sit within research, engineering heavy skills may outweigh your robust research experience. And simultaneously, some teams are more looking for exploratory researchers, so like experimental physicists, people that are studying like black holes. There's really very little concrete things that they're getting towards. They may not really need as powerful of an engineering workhorse as they need someone who's been really deep in the weeds on uncharted territory. And that's part of why residency works, is that we can support a very wide spectrum of backgrounds.

Yeah, I think that Jackie hit the nail on the head in terms of what I think as well. I think various different teams, the distribution will be slightly different based off the needs of the teams. But I do think most technical staff are some blend of research and engineering. I think this will always be true. And of course, there will be different skill sets. I'm personally more of a researcher than an engineer. Some people will have, many people on my team are more of engineers than researchers. And exactly the focus on what they do day to day, just because they care about having the most impact, will focus more on their strengths. And I think there's demand for both. To be honest, I actually think the engineering demands will probably tend to increase over the next while. And so I think there's plenty of demand for more engineers here, for sure.

Thank you, Hart. This is from Tom Burns. Is there a case of too much AI experience? For example, if someone has published at AI conferences, but on topics that are on neuroscience or less traditional AI topics? Yeah. I like to think about it this way. I think most people that I speak to today are somehow incorporating machine learning.

and AI techniques into their existing research.

So they're neuroscientists and they're leveraging our models or someone else's models.

They're chemists, you know, they're trying to leverage this domain, but the core problems that they're trying to solve, sort of what Hart alluded to, is something not ML or AI specific.
And so again, it's a little bit murky in terms of do they have too much experience or do you not?

I sit within a team of people that are evaluating everybody for research.

And so there's a lot of sharing of also names and passing of resumes, if you will.

But ultimately, if your core research work is focused on solving breakthroughs within AI and ML, I would encourage you to go towards our full-time route.

If you're exploring ways to incorporate AI and ML into your domain, I think that's a nice sweet spot that we can, that definitely pops.

Yeah, just to add on that, my background was more, I think, on the AI ML, incorporating that.

And so that's why I think my background was good for the residency program.

And I will echo that as well.

I will certainly also mention that from what I can gather of, you know, academic AI, and then also, of course, an industry applying AI to various different domains, I do think that how OpenAI and various other big AI labs approach this research will be quite different in terms of what are convincing data points in terms of which questions are being asked, just on the basis that, you know, as we scale these things up, the scope of questions that become relevant and appear before us kind of do change.
And so maybe my point is that I think that working at a big AI lab, OpenAI being very different than probably even what AI academic would be like and certainly applied AI as well.

Yeah, I also just want to come back to another point, which is we have to live in the reality that we kind of, OpenAI did, we sort of broke the mold, right?

Like, I'm talking to students who are very early in their undergraduate careers who have access to AI models and robust, meaningful data sets that they can run.

They're buying their own GPUs.

Like, we're sort of changing the forefront of when people had access than ever before.

And so residency, of course, needs to evolve to reflect the fact that people have more access to AI than ever in the history of time before.

I think if you ask some of the researchers who have PhDs in AI and ML from maybe 10 plus years ago to what we do now, it is radically different.

And so that's kind of the cool part.

And what part of the selling point of residency is, is I'm not knocking academics. I enjoyed academics, but we're also moving at such a breakneck speed that this should supplement or replace certain people's desires to go for a PhD in this area.

Thank you so much, team.

This is really valuable and inspiring.

And I want to also echo something that Hart mentioned earlier about having a non-traditional background in AI.

And I think something that you're also reiterating, Jacqueline, about us having open minds at OpenAI in that I feel that is absolutely the case.

My background is totally non-traditional for working in an AI company.

My background is in arts administration and programming and curating.

And OpenAI has a really open mind about what a diverse communion of skillsets might come together to be able to produce.

So the reason I share that is because there are a lot of questions about, well, I just finished my undergrad or I don't historically have a technical background.

And I would implore you to try, because you never know.

We do have an open mind and we're breaking all sorts of rules.

So you just never know if you don't try.

And then one of the questions that I see in the audience is, does being an OpenAI forum member give you an advantage?

And Jacqueline, I will let you give the concrete answer, but I think that- I mean, I think hearing from Hart and I directly is like somewhat beneficial.

I try to be as transparent and thoughtful when I talk to people who are interested in OpenAI and just kind of lay down the facts, which is we move so quickly that regardless of if you're a resident or a full-time member of staff, kind of going to what Hart said before, like the scrappiness factor is real and residency does need to be a priority in your journey.

And so the things to think about of when to apply is actually probably one of the biggest questions to ask yourself, because you do need to think through like, is this something that I can prioritize?

Because the scrappiness is real.

And also if you've never really dabbled in technical before, now's the time.

Like build an open source project, make some mistakes, meet some cool people in a open source community, people teach themselves languages all the time, start simple.

We do wanna see a portfolio of work.

We do wanna see some examples of research work done before.

That's really important.

We've moved beyond the flat file, as I call a resume at this point.

Like we live in a more multifaceted portfolio world.

So that is, I think if you are a forum member, you're getting kind of like the behind the curtain peak of like, we definitely need to see evidence of your technical prowess.

And maybe just to add to that as well, I would say, I mean, I actually genuinely don't know if there's like a box for if you're a forum member, you get special treatment or what have you.

But just the general, if you, we want people who are excited about all things AI and getting their hands on as many things as possible.

That's just, I think that's the position you wanna be in.

And so if we wanna, if I was to say, oh, is a person on the AI forum more likely to get an interview?

I would say yes, only on the basis that these are the kinds of people who are clearly doing what they need to do to be excited and be involved.

And really, I would encourage everyone here to really harness that and nurture that.

Because I think that that's also a skillset of just generally getting more and more passionate about the things that you care about.

I love that heart.

I really hope, I just slacked you.

We have to get a coffee.

I think you're so awesome.

And I can't believe we've been working at this place for more than a year and our paths haven't crossed.

Thank you so much for being here.

We have nine more minutes for questions and then we're gonna move into one-on-one matching networking, just to give everybody a little bit of a heads up.

This is a great question.

And Jacqueline, I get this question all the time, but is Visa sponsorship available for the residency program?

Absolutely.

So like any full-time member of staff, we can support Visa applications.

The one caveat is that if you're really pursuing a green card or basically pretty much an H-1B and beyond, we don't start that process until you convert to a full-time member of staff.

But we can absolutely support H-1B transfers.

If you're on OPT, if you need to get an O-1, if you're international, we can do that.

Just keep in mind that it adds a little bit of time and we pay for it, so it's great.

And we also offer a relocation bonus for those who don't live in the Bay Area.

It is a salary position.

The salary is $210,000 a year.

So we hope that that cost of living in San Francisco is what it is, but given the salary plus the relocation, we hope that the six months is not focused on, can I afford to live in this city, as much as you embracing the fact that you get to live here and go to work every day and sort of sit within the magical chaos that is the research team.

I'll also add, I'm not American, and so I actually went through the visa process for being a resident and it was super smooth.

So yeah, endorse everything Jackie says on that front.

Yeah.

Wow, that's really awesome.

Where are you from, Hart?

I'm from Canada.

Wow, that's super rad.

So this is from Lambert Leong.

What's the matriculation rate of residents?

And I think that means who sticks around to be full-time.

Yeah, so we have a really high conversion rate.

It's not a hundred percent and we actually don't strive to have a hundred percent.

We know that we're taking some calculated risks.

Again, we are scientists through and through.

So we even consider all hiring experimental, even for full-time.

I can say we have a hundred percent acceptance rate for those who we extend offers to.

So every resident who has been offered a chance to stay on full-time has accepted our offer to stay.

And something that I find to be the most powerful is that in the last two years of residency, not only have we converted a very high number of our residents, but some of those residents are now managers.

And in this last slate, many of our residents are now taking on mentorship role, which to me is the beauty of the program because we have people who have lived the experience of being a resident, now mentoring new people.

And it's created this incredible ecosystem of natural community building for residents to feel really safe of, okay, you have also lived this.

You understand how anxious I feel right now because I did not get the results I wanted.

And realizing that at OpenAI, if you don't get the results you wanted, that's just another data point that you've got to take and run with, not don't remorse, rejoice.

Like you got a result.

Like that's the answer.

You got a result.

And I think that feeling that you.

can get from alumni, being your mentor is amazing. I will also add, I think that you have, realistically, all of the resources in terms of matriculation rate, converting full time. I really do think it's so clear that there's so many resources invested into every resident that the goal ultimately is to convert everyone that we can. And so I think that there is certainly no concern there in terms of having access to the appropriate resources and making sure that you have an opportunity to do very, very good work. Thanks, team.

From Saif Raha, how can I gain foundational research experience relevant to the residency if I don't meet the expectations at this point in time?

Yeah, I mean, I think that the biggest thing for me, and indeed, what I did in preparation and what I would encourage basically everyone trying to ramp up into the AI space, is I do think there's not enough emphasis in just coding stuff out. And so I know I have a colleague who published Spinning Up in Deep RL. I don't know if that's still up to date, but certainly when I was applying, that would be an example of here's just a series of things you can literally code up yourself and see if it works. And I think just coming up with ideas of, oh, this would be a fun problem that can be solved with AI, turning yourself into the person who says, OK, this is a cool problem. What do I need to do to get there? And there's various different specific problems that you can find online. But my one piece of advice would be skew more towards the practical and towards the programming side of it. I do think, of course, you need the theory, but I think that that's overstated in a lot of various different resources that you'll find online. And of course, I would be remiss not to mention Carpathy has some awesome GPT videos out as well, and I think that they're great. So depending on where you are in your journey, I think that would be fantastic to look at as well. OK, awesome.

So do the residents get to pick their teams or are they placed on the team?

We work on that together. I think that's probably the most art than science part of this whole thing. At any given moment, we have teams that have a lot of bandwidth to host resident, and then sometimes teams are really locked down. They're not hiring anyone. It doesn't matter if you're full time or resident. There is just no bandwidth to support any net new people. So what I try to do during our introductory call and throughout the lifecycle of the interview process is try to understand what's going to excite you most within this space. And some people are really passionate about safety, and sometimes people are really passionate about architecture. They love doing architecture work. That's what their background is. They love systems design. And I like to tell my candidates that I want to lean into the skill sets that you have. I don't want to lean away from them. So be thoughtful in how you think about what your skill sets are going down to the first principles, foundations. I'm actually a great mathematician, and I love solving complex math problems. I love playing poker or whatever it is, something that gives me a little bit of signal of what's going to make you excited. We have a couple hundred researchers, and part of my job is kind of knowing all of them and figuring out what excites them. So knowing what Hart works on and grabbing a coffee with somebody, it's sort of threading a very fine needle because we don't take a lot of residents. And every resident that we take is very meaningful to the team. So I take it really seriously, the matching, and it should feel very exciting to you as a resident that you are being matched to a team where we think there is a great chemistry. Anything to add to that, Hart?

No, I think that pretty much covers it. I mean, in my process, I think I had a couple interviews with a couple of different teams, and these were, of course, kind of worked with the recruiter in terms of the skill set is very clearly an allocation with these teams. And so yeah, ultimately, I think that there's a little bit of an implicit discussion of like, okay, based off this background, it's pretty clear that this team is one that would be applicable to your skill set. And also, to be honest, I think before you join, it's not even obvious what teams are doing. And so I do think that this system is a good one. I certainly wouldn't be able to say before joining, oh, I wanna join this team versus that team because I don't know what any of them do.

Yeah, and I would also kind of ask for grace in that a lot of the work that our research team does is extraordinarily confidential. And even after signing NDAs, there's still limited information that we can share with you. We try our best, but we hope that you sort of take our enthusiasm and what we share with you with the knowledge that we care and that we're not taking this decision lightly. This is not something where we're just sort of bringing in a group of people and sort of like grab bagging and mapping people to a team the day before. You will know who you're working with before you arrive and you will have had great conversations with them and you will feel great about the decision. Do you echo that heart? Do you feel pretty fulfilled by the team that you're currently on?

Oh, I mean, absolutely. I actually like can't believe in terms of, I think in hindsight, it's the team that I most belong on and it's just happened this way. So yeah, good job sorting that out. Good job, Jacqueline. You're really, really awesome at your job.

Last question, cause we're at time, but so how many residents approximately a year do we accept?

Yeah, I think right now we're trending to hire a little over 30 for this year, which is about in line with what we predicted. Our finance team would like us to be a little bit more crisp in that number as they will. Generally speaking though, our residency hire is always going to be some percentage of our macro hiring trend. And so that number can sort of expand and shrink depending on what we're doing. It's gonna be in the order of double digits, but probably not more than 30 to 40 a year. And as that rolling start date is implemented, that kind of means that we can have people popping in effectively January through October. We don't have residents start in the months of November and December due to holiday schedules.

Okay. I'll actually echo or add to the point that I think the community, Jackie and everyone else who helps run the residency program, does a great job of building community within the residence. I think some of the residents here, or when we joined at the same time, some of them are like my closest friends in the city. We actually went to New Zealand together a few months ago, for example. Like, yeah, really, really strong community aspect in the residency. And I think you can build truly lifelong friendships. And I think that's a pretty universal experience from what I know.

Wow, I'm sold. I'm gonna learn to code and participate in some open source projects, guys.

Good. That was so wonderful. And I mean, this has been wonderful for me too, because Jacqueline Hart, our paths haven't crossed very intimately yet. So next time I come to San Francisco, I will put time on both of your calendars for some coffee. And thank you so much for coming, guys. It was really amazing to have you here.

Thank you very much.

Before we leave, I'm gonna share a few closing notes, and then we'll head to the one-on-one matching so that those who choose to can stick around and meet some forum members and OpenAI team members who are gonna join in the one-on-one matching. But if you're interested in applying to the residency program, we're gonna drop the link to the application now. And then I wanna remind everybody of some upcoming OpenAI forum events. Next week, August 20th, we have the virtual networking event for our non-technical OpenAI forum members. We're calling it AI in Practice, the AI enthusiasts focused on everyday solutions. So please join us for that. I'll be there again. And then Wednesday, August 28th, some of my favorite people in the forum and our inaugural forum members are the UC Berkeley Data Science for Social Justice Workshop executive director, and some of the student projects will be showcased. That's a virtual event. So please log on if you have time, 5 p.m., August 28th, 5 p.m. Pacific time.

And last but not least, if you would like to stick around and meet more forum members, please join us for one-on-one matching. To participate, you'll see the one-on-one match tab on the left-hand side of your screen, and then you'll just select join match in order to participate. Please note that you must be logged in on a laptop or computer browser to engage over video in match. We also suggest using Google Chrome or Safari browser for the best experience. Our system will pair you at random for a quick chat and will not pair you with someone you've already met during this event. And both parties must approve the match. And if you decide you don't wanna match with the person, they won't know, they won't be informed that you decided you didn't wanna match with them. Each match is set for a default of 10 minutes, but I suggest during these opportunities, if you're really trying to get to know the people in the forum, you're free to just set a timer for five minutes and then move on to the next match. You can connect with people on LinkedIn and set up a longer coffee date in the future if you meet people that you.

are really interested in getting to know a little bit better. And finally, on that note, all of our LinkedIn profiles are also embedded in our OpenAI Forum profiles. So in your chat history, you'll be able to see all of the people that showed up today for this event, and you can connect with all of us via LinkedIn. And I always love to see our OpenAI Forum members inviting me to connect on LinkedIn. So please reach out to me and others. And if you're interested in reconnecting with those folks, you can see your match history at any time in the Forum.

So have a wonderful time matching with each other. I'm so grateful to Jacqueline and Hart for meeting us here today. I want to say hello to my friend Mehul, who I haven't seen in a long time, and I wish I could have seen you face-to-face, but thanks for joining us tonight. I hope you all have a wonderful evening, and I will see you next week. Good night, everybody.