OpenAI Interest Groups

AI in Healthcare

Hi all, I've been exploring the simple-evals repo and am especially interested in the newly released HealthBench benchmark. I saw OpenAI's announcement that the dataset — 5,000 multi-turn conversations and the accompanying physician rubrics — would be open access to support the broader healthcare/LLM community.

The evaluation framework is clearly there, but I haven't been able to find the actual dataset (neither in the repo nor on Hugging Face). Is there a planned release location, or any guidance on how to access it for research and developing benchmarking?

Would love to integrate it into local workflows (e.g. evaluating Hugging Face models on-device), and happy to contribute back if there’s an effort to mirror or maintain it.


Thanks for all your work on this — it’s an incredibly valuable tool for the field.

Access to HealthBench Dataset?

The OpenAI Forum connects members to discuss AGI’s future through events, webinars, and networking. The community aims to help us achieve our mission of building AGI that benefits all of humanity.

OpenAI Forum

Job Title

Login to the OpenAI Forum community to connect with others, attend community events, and more!

Login or Sign Up for even more

To unlock all parts of the community and get the best experience, complete your profile.