Natural Language Processing

Terms

See also: Natural Language Processing terms

• attention
• bag of words
• BERT (Bidirectional Encoder Representations from Transformers)
• bigram
• bidirectional
• bidirectional language model
• BLEU (Bilingual Evaluation Understudy)
• causal language model
• crash blossom
• decoder
• denoising
• embedding layer
• embedding space
• embedding vector
• encoder
• GPT (Generative Pre-trained Transformer)
• LaMDA (Language Model for Dialogue Applications)
• language model
• large language model
• masked language model
• meta-learning
• modality
• model parallelism
• multi-head self-attention
• multimodal model
• natural language understanding
• N-gram
• NLU
• pipelining
• self-attention (also called self-attention layer)
• sentiment analysis
• sequence-to-sequence task
• sparse feature
• sparse representation
• staged training
• token
• Transformer
• trigram
• unidirectional
• unidirectional language model
• word embedding

Models

See also: Natural Language Processing Models

Model	Name	User/Org	Task	Library	Dataset	Language	Paper	Related to	License
Albert-base-v2 model	albert-base-v2		Fill-Mask	PyTorch TensorFlow JAX Safetensors Transformers Rust	Wikipedia Bookcorpus	English	1909.11942	Albert AutoTrain Compatible	Apache-2.0
Allenai/longformer-base-4096 model	longformer-base-4096	http://allenai		PyTorch TensorFlow Transformers Rust		English	2004.05150	Longformer	Apache-2.0
Allenai/scibert scivocab uncased model	scibert_scivocab_uncased	http://allenai		PyTorch JAX Transformers		English		BERT
Bert-base-cased model	bert-base-cased		Fill-Mask	PyTorch TensorFlow JAX Safetensors Transformers	Wikipedia Bookcorpus	English	1810.04805	Exbert BERT AutoTrain Compatible	Apache-2.0
Bert-base-chinese model	bert-base-chinese		Fill-Mask	PyTorch TensorFlow JAX Safetensors Transformers		Chinese	1810.04805	BERT AutoTrain Compatible
Bert-base-multilingual-cased model	bert-base-multilingual-cased		Fill-Mask	PyTorch TensorFlow JAX Safetensors Transformers	Wikipedia	104 languages	1810.04805	BERT AutoTrain Compatible	Apache-2.0
Bert-base-uncased model	bert-base-uncased		Fill-Mask	PyTorch TensorFlow JAX Safetensors Transformers Rust Core ML	Wikipedia Bookcorpus	English	1810.04805	Exbert BERT AutoTrain Compatible	Apache-2.0
Bigcode/santacoder model	santacoder	http://bigcode	Text Generation	PyTorch Transformers	Bigcode/the-stack	Code	1911.02150 2207.14255 2301.03988	Gpt2 Eval Results	Bigcode-openrail-m
Bigscience/bloom-560m model	bloom-560m	http://bigscience	Text Generation	PyTorch JAX Safetensors Transformers		48 languages	1909.08053 2110.02861 2108.12409	Bloom	Bigscience-bloom-rail-1.0
Camembert-base model	camembert-base		Fill-Mask	PyTorch TensorFlow Safetensors Transformers	Oscar	French	1911.03894	Camembert AutoTrain Compatible	Mit
Cardiffnlp/twitter-roberta-base-sentiment model	twitter-roberta-base-sentiment	http://cardiffnlp	Text Classification	PyTorch TensorFlow JAX Transformers	Tweet eval	English	2010.12421	Roberta
Cardiffnlp/twitter-roberta-base-sentiment-latest model	twitter-roberta-base-sentiment-latest	http://cardiffnlp	Text Classification	PyTorch TensorFlow Transformers	Tweet eval	English	2202.03829	Roberta
Deepset/roberta-base-squad2 model	roberta-base-squad2	http://deepset	Question Answering	PyTorch TensorFlow JAX Safetensors Transformers Rust	Squad v2	English		Roberta Eval Results AutoTrain Compatible	Cc-by-4.0
Deepset/sentence bert model	sentence_bert	http://deepset		PyTorch JAX Transformers				BERT	Apache-2.0
Distilbert-base-multilingual-cased model	distilbert-base-multilingual-cased		Fill-Mask	PyTorch TensorFlow Safetensors Transformers ONNX	Wikipedia	104 languages	1910.09700 1910.01108	Distilbert AutoTrain Compatible	Apache-2.0
Distilbert-base-uncased model	distilbert-base-uncased		Fill-Mask	PyTorch TensorFlow JAX Safetensors Transformers Rust	Wikipedia Bookcorpus	English	1910.01108	Exbert Distilbert AutoTrain Compatible	Apache-2.0
Distilbert-base-uncased-finetuned-sst-2-english model	distilbert-base-uncased-finetuned-sst-2-english		Text Classification	PyTorch TensorFlow Safetensors Transformers Rust	Sst2 Glue	English	1910.01108	Distilbert Eval Results	Apache-2.0
Distilgpt2 model	distilgpt2		Text Generation	PyTorch TensorFlow JAX Safetensors Transformers Rust TF Lite Core ML	Openwebtext	English	1910.09700 1910.01108 2201.08542 2203.12574 1503.02531	Gpt2 Exbert Eval Results Carbon Emissions	Apache-2.0
Distilroberta-base model	distilroberta-base		Fill-Mask	PyTorch TensorFlow JAX Safetensors Transformers Rust	Openwebtext	English	1910.09700 1910.01108	Exbert Roberta AutoTrain Compatible	Apache-2.0
Dslim/bert-base-NER model	bert-base-NER	http://dslim	Token Classification	PyTorch TensorFlow JAX Safetensors Transformers	Conll2003	English	1810.04805	BERT AutoTrain Compatible	Mit
Facebook/bart-large-cnn model	bart-large-cnn	http://facebook	Text2Text Generation Summarization	PyTorch TensorFlow JAX Transformers Rust	Cnn dailymail	English	1910.13461	Bart Eval Results AutoTrain Compatible	Mit
Facebook/bart-large-mnli model	bart-large-mnli	http://facebook	Text Classification Zero-Shot Classification	PyTorch JAX Safetensors Transformers Rust	Multi nli		1910.13461 1909.00161	Bart	Mit
Google/bert uncased L-2 H-128 A-2 model	bert_uncased_L-2_H-128_A-2	http://google		PyTorch JAX Transformers			1908.08962	BERT	Apache-2.0
Google/electra-base-discriminator model	electra-base-discriminator	http://google		PyTorch TensorFlow JAX Transformers Rust		English		Electra Pretraining	Apache-2.0
Gpt2 model	gpt2		Text Generation	PyTorch TensorFlow JAX Safetensors Transformers Rust TF Lite		English		Gpt2 Exbert	Mit
Gpt2-large model	gpt2-large		Text Generation	PyTorch TensorFlow JAX Safetensors Transformers Rust		English	1910.09700	Gpt2	Mit
Helsinki-NLP/opus-mt-en-es model	opus-mt-en-es	http://Helsinki-NLP	Text2Text Generation Translation	PyTorch TensorFlow JAX Transformers		English Spanish		Marian AutoTrain Compatible	Apache-2.0
Lengyue233/content-vec-best model	content-vec-best	http://lengyue233		PyTorch Transformers				Hubert	Mit
Microsoft/deberta-base model	deberta-base	http://microsoft	Fill-Mask	PyTorch TensorFlow Transformers Rust		English	2006.03654	Deberta Deberta-v1	Mit
Microsoft/layoutlm-base-uncased model	layoutlm-base-uncased	http://microsoft		PyTorch TensorFlow Transformers			1912.13318	Layoutlm
Microsoft/layoutlmv3-base model	layoutlmv3-base	http://microsoft		PyTorch TensorFlow Transformers ONNX		English	2204.08387	Layoutlmv3	Cc-by-nc-sa-4.0
Prajjwal1/bert-tiny model	bert-tiny	http://prajjwal1		PyTorch Transformers		English	1908.08962 2110.01518	BERT MNLI NLI Transformer Pre-training	Mit
Roberta-base model	roberta-base		Fill-Mask	PyTorch TensorFlow JAX Safetensors Transformers Rust	Wikipedia Bookcorpus	English	1907.11692 1806.02847	Exbert Roberta AutoTrain Compatible	Mit
Roberta-large model	roberta-large		Fill-Mask	PyTorch TensorFlow JAX Safetensors Transformers ONNX	Wikipedia Bookcorpus	English	1907.11692 1806.02847	Exbert Roberta AutoTrain Compatible	Mit
SZTAKI-HLT/hubert-base-cc model	hubert-base-cc	http://SZTAKI-HLT		PyTorch TensorFlow JAX Transformers	Wikipedia Common crawl	Hungarian		BERT	Apache-2.0
Sentence-transformers/all-MiniLM-L6-v2 model	all-MiniLM-L6-v2	http://sentence-transformers	Sentence Similarity Feature Extraction	PyTorch TensorFlow Rust Sentence Transformers	S2orc Flax-sentence-embeddings/stackexchange xml Ms marco Gooaq Yahoo answers topics Code search net Search qa Eli5 Snli Multi nli Wikihow Natural questions Trivia qa Embedding-data/sentence-compression Embedding-data/flickr30k-captions Embedding-data/altlex Embedding-data/simple-wiki Embedding-data/QQP Embedding-data/SPECTER Embedding-data/PAQ pairs Embedding-data/WikiAnswers	English	1904.06472 2102.07033 2104.08727 1704.05179 1810.09305	BERT	Apache-2.0
Sentence-transformers/all-mpnet-base-v2 model	all-mpnet-base-v2	http://sentence-transformers	Sentence Similarity Feature Extraction	PyTorch Sentence Transformers	S2orc Flax-sentence-embeddings/stackexchange xml Gooaq Yahoo answers topics Code search net Search qa Eli5 Snli Multi nli Wikihow Natural questions Trivia qa Embedding-data/sentence-compression Embedding-data/flickr30k-captions Embedding-data/altlex Embedding-data/simple-wiki Embedding-data/QQP Embedding-data/SPECTER Embedding-data/PAQ pairs Embedding-data/WikiAnswers MS Marco	English	1904.06472 2102.07033 2104.08727 1704.05179 1810.09305	Mpnet	Apache-2.0
T5-base model	t5-base		Text2Text Generation Translation Summarization	PyTorch TensorFlow JAX Safetensors Transformers Rust	C4	4 languages	1805.12471 1708.00055 1704.05426 1606.05250 1808.09121 1810.12885 1905.10044 1910.09700	T5 AutoTrain Compatible	Apache-2.0
T5-small model	t5-small		Text2Text Generation Translation Summarization	PyTorch TensorFlow JAX Safetensors Transformers Rust	C4	5 languages	1805.12471 1708.00055 1704.05426 1606.05250 1808.09121 1810.12885 1905.10044 1910.09700	T5 AutoTrain Compatible	Apache-2.0
Xlm-roberta-base model	xlm-roberta-base		Fill-Mask	PyTorch TensorFlow JAX Safetensors Transformers ONNX		94 languages	1911.02116	Exbert Xlm-roberta AutoTrain Compatible	Mit
Xlm-roberta-large model	xlm-roberta-large		Fill-Mask	PyTorch TensorFlow JAX Safetensors Transformers ONNX		94 languages	1911.02116	Exbert Xlm-roberta AutoTrain Compatible	Mit