Interface administrators, Administrators (Semantic MediaWiki), Curators (Semantic MediaWiki), Editors (Semantic MediaWiki), Suppressors, Administrators
7,785
edits
GPT: Difference between revisions
no edit summary
No edit summary Tag: Reverted |
|||
| Line 10: | Line 10: | ||
! rowspan="2" | Open Source | ! rowspan="2" | Open Source | ||
! rowspan="2" | Paper | ! rowspan="2" | Paper | ||
! rowspan="2" | Architecture | |||
|- | |- | ||
! Tokens | ! Tokens | ||
| Line 17: | Line 18: | ||
! Dataset | ! Dataset | ||
|- | |- | ||
| '''[[GPT-1]]''' || June 11, 2018 ||117 Million || 512 || 358 || <1 page || 40GB || [[BookCorpus]] || Yes || [[Improving Language Understanding by Generative Pre-Training]] | | '''[[GPT-1]]''' || June 11, 2018 ||117 Million || 512 || 358 || <1 page || 40GB || [[BookCorpus]] || Yes || [[Improving Language Understanding by Generative Pre-Training]] || | ||
|- | |- | ||
| '''[[GPT-2]]''' || February 14, 2019 || 1.5 Billion || 1024 || 716 || 1.5 pages || 40TB || [[WebText]] || Yes || [[Language Models are Unsupervised Multitask Learners]] | | '''[[GPT-2]]''' || February 14, 2019 || 1.5 Billion || 1024 || 716 || 1.5 pages || 40TB || [[WebText]] || Yes || [[Language Models are Unsupervised Multitask Learners]] || | ||
|- | |- | ||
| '''[[GPT-3]]''' || June 11, 2020 || 175 Billion || 2,048 || 1,433 || 3 pages || || [[Common Crawl]], [[WebText2]], [[Books1]], [[Books2]], [[Wikipedia]] || No || [[Language Models are Few-Shot Learners]] | | '''[[GPT-3]]''' || June 11, 2020 || 175 Billion || 2,048 || 1,433 || 3 pages || || [[Common Crawl]], [[WebText2]], [[Books1]], [[Books2]], [[Wikipedia]] || No || [[Language Models are Few-Shot Learners]] || | ||
|- | |- | ||
| '''[[GPT-3.5]]''' || March 15, 2022 || 175 Billion || 4,000 || 2,800 || 6 pages || || [[Common Crawl]], [[WebText2]], [[Books1]], [[Books2]], [[Wikipedia]] || No || | | '''[[GPT-3.5]]''' || March 15, 2022 || 175 Billion || 4,000 || 2,800 || 6 pages || || [[Common Crawl]], [[WebText2]], [[Books1]], [[Books2]], [[Wikipedia]] || No || || | ||
|- | |- | ||
| '''[[ChatGPT]]''' || November 30, 2022 || xxx || 4,096 || 2,867 || 6 pages || || [[Common Crawl]], [[WebText2]], [[Books1]], [[Books2]], [[Wikipedia]] || No || | | '''[[ChatGPT]]''' || November 30, 2022 || xxx || 4,096 || 2,867 || 6 pages || || [[Common Crawl]], [[WebText2]], [[Books1]], [[Books2]], [[Wikipedia]] || No || || | ||
|- | |- | ||
| '''[[GPT-4]]'''<br>(v1) || ???? || xxx || 8,000 || 5,600 || 12 pages || || || No || | | '''[[GPT-4]]'''<br>(v1) || ???? || xxx || 8,000 || 5,600 || 12 pages || || || No || || | ||
|- | |- | ||
| '''[[GPT-4]]'''<br>(v2) || ???? || xxx || 32,000 || 22,400 || 50 pages || || || No || | | '''[[GPT-4]]'''<br>(v2) || ???? || xxx || 32,000 || 22,400 || 50 pages || || || No || || | ||
|- | |- | ||
|} | |} | ||