Jump to content

GPT: Difference between revisions

GPT (view source)

Revision as of 09:39, 3 March 2023

26 bytes added ,  3 March 2023
no edit summary
No edit summary
No edit summary
Line 7: Line 7:
! rowspan="2" | Parameters
! rowspan="2" | Parameters
! colspan="3" | Context Window
! colspan="3" | Context Window
! colspan="7" | Training Data
! colspan="2" | Training Data
|-
|-
! Tokens
! Tokens
Line 13: Line 13:
! Equivalent
! Equivalent
! Amount
! Amount
! BookCorpus
! Dataset
! WebText
! CommonCrawl
! hello
! hello
|-
|-
| '''[[GPT-1]]''' || June 11, 2018 ||117 Million || 512 || 358 ||  || 40GB || Yes ||  ||
| '''[[GPT-1]]''' || June 11, 2018 ||117 Million || 512 || 358 ||  || 40GB || [[BookCorpus]]
|-
|-
| '''[[GPT-2]]''' ||  February 14, 2019 || 1.5 Billion || 1024 || 716 ||  || 40TB || Yes || Yes ||
| '''[[GPT-2]]''' ||  February 14, 2019 || 1.5 Billion || 1024 || 716 ||  || 40TB || [[WebText]]
|-
|-
| '''[[GPT-3]]''' || June 11, 2020 || 175 Billion || 2,048 || 1,433 ||  ||  ||  ||  ||  
| '''[[GPT-3]]''' || June 11, 2020 || 175 Billion || 2,048 || 1,433 ||  ||  ||  ||  || [[Common Crawl]], [[WebText2]], [[Books1]], [[Books2]], [[Wikipedia]]
|-
|-
| '''[[GPT-3.5]]''' || January 2022 || 1.3 Bilion, 6 Billion, 175 Billion  || 4,000 || 2,800 ||  ||  ||  ||  ||  
| '''[[GPT-3.5]]''' || January 2022 || 1.3 Bilion, 6 Billion, 175 Billion  || 4,000 || 2,800 ||  ||  ||  ||  ||  
Retrieved from "http:///wiki/Special:MobileDiff/2345"