GPQA

GPQA — Graduate-Level Google-Proof Q&A, released in 2023 — consists of multiple-choice questions in biology, physics, and chemistry written by domain PhDs and designed so that answers cannot simply be looked up. Skilled non-experts with unrestricted web access scored around 34 percent; PhD-level experts in the matching domain reached roughly 65 to 74 percent on the hardest Diamond subset.

That human baseline is what made GPQA the benchmark of the reasoning era: frontier reasoning models pushed past expert-level scores within two years of release, and GPQA Diamond became a standard line in model cards for demonstrating deep reasoning rather than recall.

Its 198-question Diamond subset is small, so a few questions move a score by whole points — differences of a point or two between models are noise, not signal.