Search results

...11, 2020 || 175 Billion || 2,048 || 1,433 || 3 pages || 570 GB || [[Common Crawl]], [[WebText2]], [[Books1]], [[Books2]], [[Wikipedia]] || No || [[Language ...5, 2022 || 175 Billion || 4,000 || 2,800 || 6 pages || 570 GB || [[Common Crawl]], [[WebText2]], [[Books1]], [[Books2]], [[Wikipedia]] || No ||

'''[[Common Crawl]]''' - open repository of web crawl data

...are both trained on unlabeled text datasets, such as Wikipedia and Common Crawl. Specifically, LaMDA is trained on dialogue training sets to generate non-g