Natural Language Processing

From AI Wiki

Terms

See also: Natural Language Processing terms

Models

See also: Natural Language Processing Models
ModelNameUser/OrgTaskLibraryDatasetLanguagePaperRelated toLicense
Albert-base-v2 modelalbert-base-v2Fill-MaskPyTorch
TensorFlow
JAX
Safetensors
Transformers
Rust
Wikipedia
Bookcorpus
English1909.11942Apache-2.0
Allenai/longformer-base-4096 modellongformer-base-4096http://allenaiPyTorch
TensorFlow
Transformers
Rust
English2004.05150Apache-2.0
Allenai/scibert scivocab uncased modelscibert_scivocab_uncasedhttp://allenaiPyTorch
JAX
Transformers
English
Bert-base-cased modelbert-base-casedFill-MaskPyTorch
TensorFlow
JAX
Safetensors
Transformers
Wikipedia
Bookcorpus
English1810.04805Apache-2.0
Bert-base-chinese modelbert-base-chineseFill-MaskPyTorch
TensorFlow
JAX
Safetensors
Transformers
Chinese1810.04805
Bert-base-multilingual-cased modelbert-base-multilingual-casedFill-MaskPyTorch
TensorFlow
JAX
Safetensors
Transformers
Wikipedia104 languages1810.04805Apache-2.0
Bert-base-uncased modelbert-base-uncasedFill-MaskPyTorch
TensorFlow
JAX
Safetensors
Transformers
Rust
Wikipedia
Bookcorpus
English1810.04805Apache-2.0
Bigcode/santacoder modelsantacoderhttp://bigcodeText GenerationPyTorch
Transformers
Bigcode/the-stackCode1911.02150
2207.14255
2301.03988
Bigcode-openrail-m
Bigscience/bloom-560m modelbloom-560mhttp://bigscienceText GenerationPyTorch
JAX
Safetensors
Transformers
48 languages1909.08053
2110.02861
2108.12409
Bigscience-bloom-rail-1.0
Camembert-base modelcamembert-baseFill-MaskPyTorch
TensorFlow
Safetensors
Transformers
OscarFrench1911.03894Mit
Cardiffnlp/twitter-roberta-base-sentiment modeltwitter-roberta-base-sentimenthttp://cardiffnlpText ClassificationPyTorch
TensorFlow
JAX
Transformers
Tweet evalEnglish2010.12421
Cardiffnlp/twitter-roberta-base-sentiment-latest modeltwitter-roberta-base-sentiment-latesthttp://cardiffnlpText ClassificationPyTorch
TensorFlow
Transformers
Tweet evalEnglish2202.03829
Deepset/roberta-base-squad2 modelroberta-base-squad2http://deepsetQuestion AnsweringPyTorch
TensorFlow
JAX
Safetensors
Transformers
Rust
Squad v2EnglishCc-by-4.0
Deepset/sentence bert modelsentence_berthttp://deepsetPyTorch
JAX
Transformers
Apache-2.0
Distilbert-base-multilingual-cased modeldistilbert-base-multilingual-casedFill-MaskPyTorch
TensorFlow
Safetensors
Transformers
ONNX
Wikipedia104 languages1910.09700
1910.01108
Apache-2.0
Distilbert-base-uncased modeldistilbert-base-uncasedFill-MaskPyTorch
TensorFlow
JAX
Safetensors
Transformers
Rust
Wikipedia
Bookcorpus
English1910.01108Apache-2.0
Distilbert-base-uncased-finetuned-sst-2-english modeldistilbert-base-uncased-finetuned-sst-2-englishText ClassificationPyTorch
TensorFlow
Safetensors
Transformers
Rust
Sst2
Glue
English1910.01108Apache-2.0
Distilgpt2 modeldistilgpt2Text GenerationPyTorch
TensorFlow
JAX
Safetensors
Transformers
Rust
TF Lite
Core ML
OpenwebtextEnglish1910.09700
1910.01108
2201.08542
2203.12574
1503.02531
Apache-2.0
Distilroberta-base modeldistilroberta-baseFill-MaskPyTorch
TensorFlow
JAX
Safetensors
Transformers
Rust
OpenwebtextEnglish1910.09700
1910.01108
Apache-2.0
Dslim/bert-base-NER modelbert-base-NERhttp://dslimToken ClassificationPyTorch
TensorFlow
JAX
Safetensors
Transformers
Conll2003English1810.04805Mit
Facebook/bart-large-cnn modelbart-large-cnnhttp://facebookText2Text Generation
Summarization
PyTorch
TensorFlow
JAX
Transformers
Rust
Cnn dailymailEnglish1910.13461Mit
Facebook/bart-large-mnli modelbart-large-mnlihttp://facebookText Classification
Zero-Shot Classification
PyTorch
JAX
Safetensors
Transformers
Rust
Multi nli1910.13461
1909.00161
Mit
Google/bert uncased L-2 H-128 A-2 modelbert_uncased_L-2_H-128_A-2http://googlePyTorch
JAX
Transformers
1908.08962Apache-2.0
Google/electra-base-discriminator modelelectra-base-discriminatorhttp://googlePyTorch
TensorFlow
JAX
Transformers
Rust
EnglishApache-2.0
Gpt2 modelgpt2Text GenerationPyTorch
TensorFlow
JAX
Safetensors
Transformers
Rust
TF Lite
EnglishMit
Gpt2-large modelgpt2-largeText GenerationPyTorch
TensorFlow
JAX
Safetensors
Transformers
Rust
English1910.09700Mit
Helsinki-NLP/opus-mt-en-es modelopus-mt-en-eshttp://Helsinki-NLPText2Text Generation
Translation
PyTorch
TensorFlow
JAX
Transformers
English
Spanish
Apache-2.0
Lengyue233/content-vec-best modelcontent-vec-besthttp://lengyue233PyTorch
Transformers
Mit
Microsoft/deberta-base modeldeberta-basehttp://microsoftFill-MaskPyTorch
TensorFlow
Transformers
Rust
English2006.03654Mit
Microsoft/layoutlm-base-uncased modellayoutlm-base-uncasedhttp://microsoftPyTorch
TensorFlow
Transformers
1912.13318
Microsoft/layoutlmv3-base modellayoutlmv3-basehttp://microsoftPyTorch
TensorFlow
Transformers
ONNX
English2204.08387Cc-by-nc-sa-4.0
Prajjwal1/bert-tiny modelbert-tinyhttp://prajjwal1PyTorch
Transformers
English1908.08962
2110.01518
Mit
Roberta-base modelroberta-baseFill-MaskPyTorch
TensorFlow
JAX
Safetensors
Transformers
Rust
Wikipedia
Bookcorpus
English1907.11692
1806.02847
Mit
Roberta-large modelroberta-largeFill-MaskPyTorch
TensorFlow
JAX
Safetensors
Transformers
ONNX
Wikipedia
Bookcorpus
English1907.11692
1806.02847
Mit
SZTAKI-HLT/hubert-base-cc modelhubert-base-cchttp://SZTAKI-HLTPyTorch
TensorFlow
JAX
Transformers
Wikipedia
Common crawl
HungarianApache-2.0
Sentence-transformers/all-MiniLM-L6-v2 modelall-MiniLM-L6-v2http://sentence-transformersSentence Similarity
Feature Extraction
PyTorch
TensorFlow
Rust
Sentence Transformers
S2orc
Flax-sentence-embeddings/stackexchange xml
Ms marco
Gooaq
Yahoo answers topics
Code search net
Search qa
Eli5
Snli
Multi nli
Wikihow
Natural questions
Trivia qa
Embedding-data/sentence-compression
Embedding-data/flickr30k-captions
Embedding-data/altlex
Embedding-data/simple-wiki
Embedding-data/QQP
Embedding-data/SPECTER
Embedding-data/PAQ pairs
Embedding-data/WikiAnswers
English1904.06472
2102.07033
2104.08727
1704.05179
1810.09305
Apache-2.0
Sentence-transformers/all-mpnet-base-v2 modelall-mpnet-base-v2http://sentence-transformersSentence Similarity
Feature Extraction
PyTorch
Sentence Transformers
S2orc
Flax-sentence-embeddings/stackexchange xml
Gooaq
Yahoo answers topics
Code search net
Search qa
Eli5
Snli
Multi nli
Wikihow
Natural questions
Trivia qa
Embedding-data/sentence-compression
Embedding-data/flickr30k-captions
Embedding-data/altlex
Embedding-data/simple-wiki
Embedding-data/QQP
Embedding-data/SPECTER
Embedding-data/PAQ pairs
Embedding-data/WikiAnswers
MS Marco
English1904.06472
2102.07033
2104.08727
1704.05179
1810.09305
Apache-2.0
T5-base modelt5-baseText2Text Generation
Translation
Summarization
PyTorch
TensorFlow
JAX
Safetensors
Transformers
Rust
C44 languages1805.12471
1708.00055
1704.05426
1606.05250
1808.09121
1810.12885
1905.10044
1910.09700
Apache-2.0
T5-small modelt5-smallText2Text Generation
Translation
Summarization
PyTorch
TensorFlow
JAX
Safetensors
Transformers
Rust
C45 languages1805.12471
1708.00055
1704.05426
1606.05250
1808.09121
1810.12885
1905.10044
1910.09700
Apache-2.0
Xlm-roberta-base modelxlm-roberta-baseFill-MaskPyTorch
TensorFlow
JAX
Safetensors
Transformers
ONNX
94 languages1911.02116Mit
Xlm-roberta-large modelxlm-roberta-largeFill-MaskPyTorch
TensorFlow
JAX
Safetensors
Transformers
ONNX
94 languages1911.02116Mit