June 30, 2026 — OpenAI introduces GeneBench-Pro, a research-level benchmark measuring how AI agents navigate ambiguity and make consequential judgments in computational biology.
Scientific data rarely arrive with instructions. Researchers must decide whether a pattern reflects biology or noise, whether the data can support the question being asked, and how each result should change what they do next. AI agents are increasingly capable of executing complex analyses, but real scientific research also depends not simply on recalling facts or following a predefined workflow but also on making these higher-order judgments.
GeneBench-Pro expands on GeneBench to cover harder, more realistic tasks across genomics, quantitative biology, and translational medicine, capturing the complexity, iterative nature, and ambiguity of scientific research in computational biology.
Within GeneBench-Pro, OpenAI defines “research taste” as the chains of judgment calls that shape an analysis: which questions the data can support, how early diagnostics should change the model or estimand, and when an initial plan needs to be revised. Each problem gives the model a realistic and messy dataset, brief experimental context, and a target estimand tied to a downstream decision. To answer correctly, the model must explore the data, choose an appropriate analytical approach, engage in an iterative process of experimentation, and supply a final answer.
Dataset Construction
GeneBench-Pro includes 129 questions covering a broad range of computational biology settings and methods across 10 domains and 21 sub-domains.
Each problem is built synthetically: OpenAI knows the full causal structure and directly simulates the data-generating process. That enables tuning complexity, ensuring reasonable differences in subjective analytical choices still produce accepted numerical results, and verifying through ablation studies that plausible but incorrect analyses fail.
OpenAI sent 82 of the 129 questions to external domain experts, including graduate students, postdoctoral researchers, industry scientists, and professors. Reviewers assessed each problem’s realism, whether the target answer was identifiable, and whether the methods and estimators were appropriate.
Alexander Strudwick Young, Assistant Professor in Human Genetics at UCLA: “The problems I reviewed would have been challenging for a graduate student to complete without iterated feedback from an experienced supervisor. The data contained technical and quality control issues that required thoughtful and reflective data analysis.”
Evaluation and Grading
Each GeneBench-Pro problem is a self-contained scientific analysis. Agents receive access to an isolated workspace with a short prompt, data files, and a standard bioinformatics stack including Python, scientific computing libraries, and basic genomics packages like PLINK 2.0.
Because OpenAI controls the full data-generation process, it can grade correctness deterministically against known targets, avoiding model-choice variability and verbosity effects found in standard rubric-based evaluation.
OpenAI is fully open-sourcing 10 representative GeneBench-Pro questions on Hugging Face, with an interactive web interface for browsing them. A 50-question subset will be provided to Artificial Analysis for independent, third-party benchmarking.
Results
GPT-5.6 Sol attains a pass rate of 28.7% at the highest reasoning level (31.5% with Pro mode enabled). That is a sharp increase from when OpenAI began building the original GeneBench; at that time, GPT-5 scored below 5%. At the current pace, this benchmark may be saturated by the end of the year.
At the lowest reasoning level, GPT-5.6 Sol only achieves a single-digit pass rate. At the highest reasoning level, GPT-5.6 Sol solves nearly six times as many questions as GPT-5.2 does while using about two-thirds as many tokens.
Competitor models at best matched the performance of the corresponding GPT model at the time of release. Opus 4.8 achieved 16.0%, Gemini 3.5 Flash scored 8.1%, Gemini 3.1 Pro scored 3.1%, Grok 4.3 scored 1.5%, GLM 5.2 scored 4.6%, and DeepSeek V4 Pro scored 2.4%.
Reviewers estimated that a typical GeneBench-Pro problem would take a human expert around 20–40 hours to complete. At a conservative $200 per hour, that puts the human labor cost of a single problem in the thousands of dollars. Current inference costs are only several dollars per problem.
GeneBench-Pro represents an initial effort to evaluate the more abstract skills involved in good scientific judgment possessed by experienced researchers — skills that allow them to intuit promising initial analyses, iterate when data contradict assumptions, and arrive at conclusions upon which downstream clinical, academic, or business decisions may depend.