Interface administrators, Administrators (Semantic MediaWiki), Curators (Semantic MediaWiki), Editors (Semantic MediaWiki), Suppressors, Administrators
7,785
edits
GPT: Difference between revisions
no edit summary
No edit summary |
No edit summary |
||
| Line 7: | Line 7: | ||
! rowspan="2" | Parameters | ! rowspan="2" | Parameters | ||
! colspan="3" | Context Window | ! colspan="3" | Context Window | ||
! colspan=" | ! colspan="2" | Training Data | ||
|- | |- | ||
! Tokens | ! Tokens | ||
| Line 13: | Line 13: | ||
! Equivalent | ! Equivalent | ||
! Amount | ! Amount | ||
! | ! Dataset | ||
|- | |- | ||
| '''[[GPT-1]]''' || June 11, 2018 ||117 Million || 512 || 358 || || 40GB || | | '''[[GPT-1]]''' || June 11, 2018 ||117 Million || 512 || 358 || || 40GB || [[BookCorpus]] | ||
|- | |- | ||
| '''[[GPT-2]]''' || February 14, 2019 || 1.5 Billion || 1024 || 716 || || 40TB || | | '''[[GPT-2]]''' || February 14, 2019 || 1.5 Billion || 1024 || 716 || || 40TB || [[WebText]] | ||
|- | |- | ||
| '''[[GPT-3]]''' || June 11, 2020 || 175 Billion || 2,048 || 1,433 || || || || || | | '''[[GPT-3]]''' || June 11, 2020 || 175 Billion || 2,048 || 1,433 || || || || || [[Common Crawl]], [[WebText2]], [[Books1]], [[Books2]], [[Wikipedia]] | ||
|- | |- | ||
| '''[[GPT-3.5]]''' || January 2022 || 1.3 Bilion, 6 Billion, 175 Billion || 4,000 || 2,800 || || || || || | | '''[[GPT-3.5]]''' || January 2022 || 1.3 Bilion, 6 Billion, 175 Billion || 4,000 || 2,800 || || || || || | ||