GPT-4
- See also: GPT-4 Plugins
Exams
| Exam | GPT-4 Points | GPT-4 Percentile | GPT-4 (no vision) Points | GPT-4 (no vision) Percentile | GPT-3.5 Points | GPT-3.5 Percentile |
|---|---|---|---|---|---|---|
| Uniform Bar Exam (MBE+MEE+MPT)1 | 298 / 400 | ~90th | 298 / 400 | ~90th | 213 / 400 | ~10th |
| LSAT | 163 | ~88th | 161 | ~83rd | 149 | ~40th |
| SAT Evidence-Based Reading & Writing | 710 / 800 | ~93rd | 710 / 800 | ~93rd | 670 / 800 | ~87th |
| SAT Math | 700 / 800 | ~89th | 690 / 800 | ~89th | 590 / 800 | ~70th |
| Graduate Record Examination (GRE) Quantitative | 163 / 170 | ~80th | 157 / 170 | ~62nd | 147 / 170 | ~25th |
| Graduate Record Examination (GRE) Verbal | 169 / 170 | ~99th | 165 / 170 | ~96th | 154 / 170 | ~63rd |
| Graduate Record Examination (GRE) Writing | 4 / 6 | ~54th | 4 / 6 | ~54th | 4 / 6 | ~54th |
| USABO Semifinal Exam 2020 | 87 / 150 | 99th–100th | 87 / 150 | 99th–100th | 43 / 150 | 31st–33rd |
| USNCO Local Section Exam 2022 | 36 / 60 | 38 / 60 | 24 / 60 | |||
| Medical Knowledge Self-Assessment Program | 75% | 75% | 53% | |||
| Codeforces Rating | 392 | below 5th | 392 | below 5th | 260 | below 5th |
| AP Art History | 5 | 86th–100th | 5 | 86th–100th | 5 | 86th–100th |
| AP Biology | 5 | 85th–100th | 5 | 85th–100th | 4 | 62nd–85th |
| AP Calculus BC | 4 | 43rd–59th | 4 | 43rd–59th | 1 | 0th–7th |
Benchmarks
| Benchmark | GPT-4 | Evaluated few-shot | GPT-3.5 | Evaluated few-shot | LM SOTA | Best external LM evaluated few-shot | SOTA | Best external model (includes benchmark-specific training) |
|---|---|---|---|---|---|---|---|---|
| MMLU | 86.4% | 5-shot | 70.0% | 5-shot | 70.7% | 5-shot U-PaLM | 75.2% | 5-shot Flan-PaLM |
| HellaSwag | 95.3% | 10-shot | 85.5% | 10-shot | 84.2% | LLAMA (validation set) | 85.6% | ALUM |
| AI2 Reasoning Challenge (ARC) | 96.3% | 25-shot | 85.2% | 25-shot | 84.2% | 8-shot PaLM | 85.6% | ST-MOE |
| WinoGrande | 87.5% | 5-shot | 81.6% | 5-shot | 84.2% | 5-shot PALM | 85.6% | 5-shot PALM |
| HumanEval | 67.0% | 0-shot | 48.1% | 0-shot | 26.2% | 0-shot PaLM | 65.8% | CodeT + GPT-3.5 |
| DROP (f1 score) | 80.9 | 3-shot | 64.1 | 3-shot | 70.8 | 1-shot PaLM | 88.4 |
Benmarks (Visual)
| Benchmark | GPT-4 | Evaluated few-shot | Few-shot SOTA | SOTA | Best external model (includes benchmark-specific training) | |
|---|---|---|---|---|---|---|
| VQAv2 | 77.2% | 0-shot | 67.6% | Flamingo 32-shot | 84.3% | PaLI-17B |
| TextVQA | 78.0% | 0-shot | 37.9% | Flamingo 32-shot | 71.8% | PaLI-17B |
| ChartQA | 78.5%A | - | 58.6% | Pix2Struct Large | - | |
| AI2 Diagram (AI2D) | 78.2% | 0-shot | - | 42.1% | Pix2Struct Large | - |
| DocVQA | 88.4% | 0-shot (pixel-only) | - | 88.4% | ERNIE-Layout 2.0 | - |
| Infographic VQA | 75.1% | 0-shot (pixel-only) | - | 61.2% | Applica.ai TILT | - |
| TVQA | 87.3% | 0-shot | - | 86.5% | MERLOT Reserve Large | - |
| LSMDC | 45.7% | 0-shot | 31.0% | MERLOT Reserve 0-shot | 52.9% | MERLOT |