GPT

GPT, short for Generative Pre-trained Transformer, is the name of a model family developed by OpenAI. The first model, now commonly called GPT-1, combined a Transformer language model with generative pre-training on unlabeled text and supervised fine-tuning for individual language tasks. GPT-2 shifted attention toward zero-shot task performance, and GPT-3 showed that a much larger model could perform many tasks from instructions or demonstrations placed in its prompt. Later systems added instruction tuning, reinforcement learning from human feedback, multimodal input, tool use, and models that spend additional computation on reasoning.^{[1][2][4][11][13][16][17][20][21]}

The published record does not support treating every GPT generation as a simple scale-up of one fully disclosed design. OpenAI documented the model size and architecture of GPT-1, GPT-2, and GPT-3, but its GPT-4 technical report withheld model size, hardware, training compute, dataset construction, and training-method details. OpenAI has likewise not published parameter counts for GPT-4 or later general-purpose GPT models.^[5] Claims that GPT-4 or later models have a particular number of parameters, experts, layers, or modality-specific encoders are therefore unconfirmed unless OpenAI publishes those details.

GPT can also mean several related but distinct things. It can refer to OpenAI's model lineage, to a model patterned after the early generative pre-trained Transformer papers, or to a customized ChatGPT application called a "GPT." It is not a precise synonym for every decoder-only large language model. Other organizations have used GPT in model names, but similar models are also released under unrelated family names. This article uses GPT primarily for OpenAI's model family and identifies other uses explicitly.

Name and scope

The phrase "generative pre-training" describes the use of a generative language model objective during pre-training. In the 2018 paper, the model first learned from unlabeled text by estimating the probability of each token from preceding tokens. It was then fine-tuned with labeled examples for tasks such as textual entailment, question answering, semantic similarity, and classification. The authors presented this as a way to transfer a common learned representation to multiple tasks without building a different neural architecture for each one.^[2]

The word "Transformer" comes from the architecture introduced by Vaswani and colleagues in 2017. Their original system was an encoder-decoder network for sequence transduction. Its decoder used masking so that a position could not attend to later positions while predicting the next output token.^[1] GPT-1 adapted the decoder side into a stand-alone, left-to-right language model. That historical relationship does not mean that the original Transformer and every GPT have the same complete architecture.

OpenAI later used "GPTs" as a product term for customized versions of ChatGPT. Announced in November 2023, these applications can combine persistent instructions, uploaded knowledge, and enabled tools. They are configurations around models and product services, not separately pre-trained foundation models. The GPT Store, launched in January 2024, provided a directory for sharing them; OpenAI said users had created more than three million custom GPTs by the store's launch.^[6][7]

OpenAI filed a United States trademark application for the word mark GPT, serial number 97733259. The procedural record is more qualified than a claim that a court or the USPTO conclusively ruled the term generic. The USPTO's Trademark Trial and Appeal Board system records an ex parte appeal filed in August 2024, an acknowledged appeal and remand, and an application status of "suspension letter - mailed." As of the research cutoff for this article, that proceeding did not justify describing the trademark dispute as finally resolved.^[8]

Development history

GPT-1

OpenAI announced its first generative pre-training work on June 11, 2018. The accompanying paper described a 12-layer, decoder-only Transformer with masked self-attention, a 768-dimensional state, 12 attention heads, learned position embeddings, and sequences of 512 tokens. The paper's experiments used BooksCorpus, which it described as more than 7,000 unpublished books with long passages of continuous text. The model was pre-trained as a language model and then fine-tuned on supervised tasks with task-aware input transformations.^[2][3]

The paper did not yet establish the later practice of using a single prompted model for most tasks. Its main results depended on updating the model for each supervised dataset. OpenAI's associated post also emphasized limitations: the pre-training step took about a month on eight GPUs, text alone gave an incomplete and biased view of the world, and generalization remained brittle under systematic, adversarial, or out-of-distribution evaluation.^[3]

The model is usually reported as having about 117 million parameters. That figure appears as the smallest architecture in the GPT-2 paper's comparison table, where the 117 million parameter model corresponds to the original GPT configuration.^[4]

GPT-2

OpenAI announced GPT-2 on February 14, 2019. Its largest version had 1.5 billion parameters, 48 layers, a 1,600-dimensional model state, and a context size of 1,024 tokens. The family also included 117 million, 345 million, and 762 million parameter versions. GPT-2 used byte-level byte-pair encoding and largely followed GPT-1 while changing layer normalization, initialization, vocabulary size, and context length.^[4][9]

GPT-2 was trained on WebText, a corpus assembled from outbound Reddit links that had received at least three karma. After deduplication and heuristic cleaning, the paper reported slightly more than eight million documents and 40 GB of text; Wikipedia documents were removed to reduce overlap with common evaluation data. GPT-2's objective remained next-token prediction, but the paper tested whether tasks demonstrated in ordinary text could be performed without task-specific gradient updates. It reported zero-shot results across language modeling and several other tasks, while also saying that many downstream scores remained below specialized systems.^[4]

OpenAI initially released only a smaller model because of concerns about automated deceptive or abusive text. It subsequently used a staged release process and published the full 1.5 billion parameter model and weights on November 5, 2019. OpenAI described the process as an experiment in responsible publication rather than proof that a particular release policy would fit every later model.^[9][10]

GPT-3 and in-context learning

The GPT-3 paper appeared in May 2020, and OpenAI announced a commercial API in private beta on June 11, 2020. GPT-3 was an autoregressive model with 175 billion parameters, 96 layers, a 12,288-dimensional model state, 96 attention heads, and a 2,048-token context window. The paper also evaluated seven smaller sizes, from 125 million to 13 billion parameters.^[11][12]

GPT-3's best-known contribution was a systematic evaluation of zero-shot, one-shot, and few-shot prompting. In the paper's terminology, few-shot evaluation provided a task description and several examples in context, but did not update model weights. One-shot used one example, and zero-shot used only the task instruction. This behavior is now commonly discussed as in-context learning. GPT-3 improved on many tasks as model size increased, but the authors also documented weak results on some tasks, sensitivity to the prompt, bias, and contamination risks when web training corpora contain benchmark material.^[11]

Two GPT-3 data figures are easy to conflate. The source corpora listed in the paper contained about 410 billion filtered Common Crawl tokens, 19 billion WebText2 tokens, 12 billion Books1 tokens, 55 billion Books2 tokens, and 3 billion English Wikipedia tokens. These quantities total about 499 billion tokens. Training did not consume that entire mixture once in proportion to its size. The paper states that each model was trained for 300 billion sampled tokens, with higher-quality corpora deliberately sampled more often.^[11] Describing GPT-3 as simply "trained on 400 billion tokens" therefore obscures the distinction between corpus size and tokens actually sampled during training.

The 2020 API exposed models through a hosted service instead of releasing GPT-3 weights. OpenAI presented a general text-in, text-out interface so developers could use the same family for different tasks, with optional task-specific fine-tuning added later. This deployment model made model behavior, versioning, and service features separate from the underlying research model.^[12]

Instruction following and ChatGPT

Raw next-token prediction does not ensure that a model follows a user's intent. The InstructGPT work addressed that mismatch using human feedback. Its published procedure had three steps: supervised fine-tuning on human-written demonstrations, training a reward model from human rankings of model outputs, and optimizing the language-model policy against that reward model with reinforcement learning. The study trained 1.3 billion, 6 billion, and 175 billion parameter models based on GPT-3.^[13]

On the study's distribution of prompts, human evaluators preferred outputs from the 1.3 billion parameter InstructGPT model to outputs from the original 175 billion parameter GPT-3. This was a result about preference and instruction following on the evaluated prompts, not evidence that alignment compressed all language capability by a factor of 100. The paper also reported remaining mistakes, limited improvements on some bias evaluations, and performance regressions on some public tasks unless the training procedure mixed in pre-training data.^[13]

ChatGPT launched as a free research preview on November 30, 2022. OpenAI described it as a conversational sibling to InstructGPT and said it was fine-tuned from a model in the GPT-3.5 series that had finished training in early 2022. The dialogue format was designed to support follow-up questions, correction of premises, and refusal behavior. The launch post did not disclose a GPT-3.5 parameter count, so estimates that assign it the 175 billion parameter count of the largest GPT-3 model should not be presented as confirmed.^[14]

GPT-4, GPT-4 Turbo, and GPT-4o

GPT-4 was released on March 14, 2023. OpenAI described it as a large multimodal model that accepted text and image inputs and produced text. Its base model was pre-trained to predict the next token using public and licensed data and was then post-trained with reinforcement learning from human feedback. The technical report highlighted performance on academic and professional examinations, but also warned that GPT-4 could hallucinate, make reasoning errors, and be confidently wrong.^[5][15]

The GPT-4 report deliberately omitted model size, hardware, training compute, dataset construction, and full training-method details. It called GPT-4 a Transformer-style model but did not validate circulated claims that it had about 1.8 trillion total parameters or a particular mixture-of-experts layout. Those claims should not appear in a factual lineage table as if they were published specifications.^[5]

GPT-4 Turbo was announced at OpenAI DevDay on November 6, 2023. The preview model offered a 128,000-token context window, compared with the smaller context variants available with the original GPT-4 service, and OpenAI said its knowledge of world events extended to April 2023. The same announcement expanded API support for function calling, JSON output, and vision. These are product and interface features; their presence does not disclose the model's internal parameter count or architecture.^[16]

GPT-4o, announced on May 13, 2024, extended the family across audio, vision, and text. OpenAI said the "o" stood for "omni," and described a single model trained end to end across text, vision, and audio. It accepts combinations of text, audio, image, and video and can generate text, audio, and image outputs. The system card characterized it as autoregressive and reported audio response latency as low as 232 milliseconds and 320 milliseconds on average under the tested setup.^[17][18] This published description contradicts a generic claim that GPT-4o necessarily uses separately disclosed modality encoders and pipelines. OpenAI has not published that level of architectural detail.

GPT-4.1 and the reasoning line

OpenAI launched GPT-4.1, GPT-4.1 mini, and GPT-4.1 nano in the API on April 14, 2025. All three supported up to one million tokens of context. The launch focused on coding, instruction following, and long-context use. A context limit is a service specification, not a guarantee that a model will retrieve or reason over every item in a very long input with uniform accuracy.^[19]

The GPT name did not cover every OpenAI language model during this period. In September 2024, OpenAI introduced the separate OpenAI o1 reasoning line. OpenAI reported that o1's performance improved with more reinforcement-learning compute during training and more time spent reasoning during inference. The o-series work is relevant to later GPT systems because GPT-5 combined fast and deeper-reasoning behavior, but o1, o3, and o4-mini were not GPT releases.^[20]

GPT-5 and point releases

GPT-5 launched on August 7, 2025. In ChatGPT, OpenAI described it as a unified system with a fast model, a deeper reasoning model, and a real-time router that selected between them based on the task and user intent. GPT-5 Pro used additional test-time computation for difficult prompts. In the API, OpenAI released GPT-5, GPT-5 mini, and GPT-5 nano with controls for reasoning effort and response verbosity.^[21][22]

The original GPT-5 context specification is often misstated. OpenAI's developer announcement says the API models accepted at most 272,000 input tokens and could emit at most 128,000 reasoning and output tokens, for a total context length of 400,000 tokens.^[22] The 272,000 input maximum should not be rounded down to 256,000, and the 400,000 total should not be described as 400,000 input tokens.

OpenAI then used a relatively rapid point-release sequence:

• GPT-5.1 entered the API on November 13, 2025. OpenAI described adaptive reasoning, a no-reasoning mode, and up to 24-hour prompt caching.^[23]
• GPT-5.2 launched on December 11, 2025 in Instant, Thinking, and Pro forms in ChatGPT, with corresponding API models. OpenAI explicitly cautioned that the models remained imperfect and that critical answers should be checked.^[24]
• GPT-5.3 Instant launched on March 3, 2026 for ChatGPT and as gpt-5.3-chat-latest in the API. OpenAI said Thinking and Pro updates would follow rather than calling this an all-variant GPT-5.3 release.^[25]
• GPT-5.4 launched on March 5, 2026 in ChatGPT as GPT-5.4 Thinking, in the API, and in Codex; GPT-5.4 Pro also launched in ChatGPT and the API.^[26]
• GPT-5.5 Thinking and Pro began rolling out on April 23, 2026, and API availability followed on April 24. The API model offered a one-million-token context window. GPT-5.5 Instant followed on May 5 and replaced GPT-5.3 Instant as ChatGPT's default model for logged-in users.^[27][28]
• GPT-5.6 launched on July 9, 2026 as a three-tier family: Sol, Terra, and Luna. OpenAI made the family available across ChatGPT, Codex, and the API; its launch announcement did not disclose parameter counts.^[42]

OpenAI's January 29, 2026 retirement notice said GPT-4o, GPT-4.1, GPT-4.1 mini, and o4-mini would leave ChatGPT on February 13, along with previously announced retirements of GPT-5 Instant and Thinking. The notice said there was no corresponding API change at that time.^[29] Product retirement does not erase the historical model release and does not imply that all applications using an older API snapshot stopped at the same date.

As of the July 28, 2026 research cutoff used for this article, GPT-5.6 was the latest released GPT generation covered here. Material first published after that cutoff is intentionally excluded.^[42]

Release summary

Scroll sideways for more →

Dates in this table refer to a paper, announcement, API launch, or product rollout as specified. They should not be interpreted as a single uniform meaning of "release." OpenAI has often rolled models out gradually and has used different names and availability rules in ChatGPT and the API.

Architecture and objective

Documented early architecture

GPT-1, GPT-2, and GPT-3 are documented as autoregressive, decoder-only Transformers. At each position, masked or causal self-attention prevents the model from using later tokens to predict an earlier token. Each block combines attention with a position-wise feed-forward network, residual connections, and normalization. Embeddings convert discrete tokens and positions into vectors, and an output projection produces a probability distribution over the vocabulary.^{[1][2][4][11]}

For a token sequence x1, x2, ..., xn, an autoregressive model factorizes its probability as the product of conditional probabilities P(xi | x1, ..., x(i-1)). Training minimizes the negative log probability assigned to the observed next tokens. Because the target comes from the original sequence, this is a form of self-supervised learning rather than a process that requires a separate human label for every token.^[2][4]

The three early generations changed more than parameter count:

• GPT-1 used a 12-layer decoder, 12 heads, a 768-dimensional state, learned positional embeddings, and a 40,000-merge byte-pair vocabulary.^[2]
• GPT-2's largest model used 48 layers and a 1,600-dimensional state. It adopted byte-level byte-pair encoding with a 50,257-token vocabulary and expanded the context from 512 to 1,024 tokens.^[4]
• GPT-3's 175 billion parameter model used 96 layers, a 12,288-dimensional state, and alternating dense and locally banded sparse attention patterns. All eight sizes in its study used a 2,048-token context.^[11]

These specifications come from the relevant papers and should not be projected onto later models. OpenAI's GPT-4 report only says that GPT-4 is Transformer-style and pre-trained to predict the next token. GPT-4o's public materials describe an autoregressive omni model trained end to end across text, vision, and audio. OpenAI has not published enough detail to say whether GPT-4, GPT-5, or their point releases use a particular mixture-of-experts scheme, layer count, attention variant, or dedicated encoder arrangement.^[5][18]

Tokens and context

A model processes tokens rather than characters or words directly. A token can represent a whole short word, part of a word, punctuation, whitespace, or a byte sequence. GPT-2's byte-level byte-pair encoding was designed to represent arbitrary Unicode text without an unmanageably large base vocabulary. Tokenization affects the number of positions a passage consumes and can differ substantially among languages and kinds of text.^[4]

The context window is the maximum token budget for the material a model can process in one request under a specified service configuration. Depending on the model, that budget may include input, output, and internal reasoning tokens. It is not the same as the model's training corpus, knowledge cutoff, persistent product memory, or guaranteed recall. A one-million-token context allows a large input to be supplied, but accuracy can still depend on where relevant material appears, how much distracting material is present, and the task being evaluated.^[19]

Generation

At inference time, a text model repeatedly selects or samples a token from its predicted distribution and appends it to the context. Decoding settings can make output more deterministic or more varied. The generated text remains conditioned on the prompt and preceding output, but it is not retrieved verbatim from a database by default. Product systems can separately add retrieval-augmented generation, web search, code execution, or other tools. A claim that "GPT can browse" or "GPT can run code" is therefore incomplete unless it identifies the deployed product and enabled tools.^[4][16][22]

Training and post-training

Pre-training data

The best-documented training mixtures are those for GPT-1 through GPT-3. GPT-1 used BookCorpus. GPT-2 used WebText, assembled from selected outbound Reddit links. GPT-3 used filtered Common Crawl, WebText2, two internet-book corpora, and English Wikipedia, with deliberate reweighting of higher-quality sources.^[2][4][11]

OpenAI says GPT-4 used publicly available and licensed third-party data, but the technical report does not disclose a full source list or corpus size.^[5] The public record does not support filling that gap with assumptions that a specific undisclosed corpus, amount of synthetic data, or code repository must have been used. Later system cards describe broad classes of data, but they do not provide a complete, independently auditable inventory of every training item.

Data preparation can include filtering, deduplication, quality scoring, and removal of known evaluation material. These procedures cannot guarantee that a web-scale corpus is accurate, unbiased, free of personal information, or uncontaminated by benchmark content. GPT-3's authors found that a filtering bug left some benchmark overlap and discussed the possible effect rather than treating benchmark scores as automatically independent of training data.^[11]

Supervised and preference-based post-training

Pre-training creates a general next-token model. Post-training adapts behavior to instructions, conversations, safety policies, tools, or a target domain. InstructGPT provides a published example. Human labelers wrote preferred answers, ranked candidate responses, and supplied the comparisons used to train a reward model. The team then optimized a supervised policy against that learned reward using reinforcement learning.^[13]

Reinforcement learning from human feedback can make outputs more helpful on the distribution represented by the feedback, but it does not turn a preference judgment into a factuality proof. Labelers can disagree, reward models can be imperfect, and optimization can reduce performance on tasks that are not represented in the reward. The InstructGPT paper documented improvements in preference, truthfulness, and toxicity measures alongside remaining errors and limited improvement on some bias datasets.^[13]

OpenAI has changed its post-training methods over time. The GPT-5 launch described "safe completions," in which the system aims to provide as much useful information as possible while respecting safety limits, instead of treating every difficult request as a binary comply-or-refuse choice.^[21] Public descriptions of a model's policy behavior should not be expanded into claims about exact proprietary loss functions or hidden training datasets.

Reasoning and inference-time computation

Prompt engineering changes the information and instructions placed in context. Few-shot examples can specify a task without modifying weights, and prompting a model to produce intermediate reasoning can improve some multi-step tasks. The 2022 chain-of-thought study reported gains on arithmetic, commonsense, and symbolic reasoning tasks for sufficiently large models, but this was an empirical prompting result rather than a guarantee for every model or problem.^[31]

OpenAI's o1 work introduced a stronger distinction between ordinary answer generation and models trained to spend more computation on an internal reasoning process. OpenAI reported smooth gains with additional reinforcement-learning compute and test-time thinking in its evaluated settings.^[20] GPT-5 later exposed a system that could route between a fast response and deeper reasoning. The precise internal chain of thought is not generally shown to users, and a longer reasoning process can still end in an incorrect conclusion.

Scaling laws and compute allocation

Research associated with early GPT development found power-law relationships between language-model loss and model size, dataset size, and training compute across the tested ranges. Kaplan and colleagues concluded that large models were more sample-efficient and, under their assumptions, recommended allocating compute toward larger models trained on relatively fewer tokens.^[32]

The Chinchilla study revisited that allocation with more than 400 model runs. It concluded that compute-optimal model size and training-token count should scale in roughly equal proportion. Its 70 billion parameter Chinchilla model used the same stated compute budget as the 280 billion parameter Gopher while training on 1.4 trillion tokens, and it outperformed several larger models on the paper's evaluation suite.^[33]

These papers describe empirical relationships for particular architectures, datasets, and compute regimes. They do not reveal the undisclosed size or training-token count of later GPT models. It is reasonable to use them to explain why both data and parameter allocation matter, but not to assert that GPT-4 or GPT-5 follows one exact published ratio.

Interaction and deployment

Prompting and in-context learning

In-context learning lets a model infer a requested pattern from instructions and examples in the prompt. It is different from fine-tuning because the examples do not permanently update model weights. The effect ends when that context is no longer supplied, unless a product stores and reuses information separately. Prompt wording, example choice, order, formatting, and irrelevant context can all alter the result.^[11]

Few-shot prompting can reduce the need for a labeled training set for a one-off task, but it uses context capacity and may be unstable. Fine-tuning changes model behavior across requests and can be preferable when a task requires repeated consistency. Retrieval supplies external documents at inference time. Tool calls let the system ask software or services to take an action. These mechanisms can be combined, but they solve different problems.

ChatGPT and the API

ChatGPT is a consumer and enterprise application that has used multiple GPT and o-series models. Its features, model picker, memory, search, file handling, voice mode, and usage limits can change without changing the historical identity of a base model. An API model name, by contrast, identifies a hosted developer endpoint or snapshot with its own input limits, pricing, and deprecation policy.

The distinction matters for retirement notices. When OpenAI removed GPT-4o and several older choices from ChatGPT in February 2026, it explicitly said there was no API change at that time.^[29] Statements about availability should therefore name the surface, date, plan, and model identifier whenever possible.

OpenAI reported on February 27, 2026 that more than 900 million people used ChatGPT each week. That is a company-reported product metric, not a count of distinct people who directly selected one particular GPT model or an independently audited measure of model quality.^[34]

Tools and agents

Modern GPT deployments can generate structured function arguments, call external functions, search indexed files, use web search, and interact with code or computer environments when those capabilities are enabled. The model proposes an action, while the surrounding application validates, executes, and returns the result. Security therefore depends on the complete system, including permissions, tool definitions, data boundaries, confirmation rules, and treatment of untrusted instructions.^{[16][22][40][41]}

Tool use can improve factual grounding or allow a model to act on current information, but it introduces new failures. A model can select the wrong tool, pass incorrect arguments, misunderstand a result, or follow malicious instructions embedded in retrieved content. Developers should distinguish model errors from execution-layer errors and should not grant a text model broader authority than the task requires.^[40][41]

Capabilities and evaluation

GPT models can generate and transform text, summarize supplied documents, answer questions, translate, classify, extract structured data, and produce or analyze code. Multimodal variants can also process images and, in GPT-4o deployments, audio and video inputs. Tools can extend this to retrieval, calculations, software actions, and image generation.^[15][17]

Capability depends on the exact model, prompt, language, domain, tool configuration, and evaluation protocol. A score reported by OpenAI is evidence about the tested model under the stated conditions, not a permanent rank over every competitor or a guarantee for users. Important details include whether tools were enabled, whether multiple samples or reranking were used, the reasoning-effort setting, the benchmark version, and possible training contamination.

The GPT-4 report illustrates both the value and limits of benchmark evidence. It reported strong results on many exams and said GPT-4 scored around the top 10 percent on a simulated bar exam. The same report warned that GPT-4 was not fully reliable and could hallucinate or make reasoning errors.^[5] A benchmark can show that a model answers one defined set of questions well while saying little about factual accuracy on changing, ambiguous, or adversarial real-world requests.

GPT-3's contamination analysis is also relevant to later evaluation. When benchmarks are available on the public web, their questions, answers, or close paraphrases may enter a training corpus. Deduplication and overlap checks reduce that risk but may miss transformations or private benchmark copies. Independent evaluations should use held-out or newly created tasks where possible and document the prompts and scoring method.^[11]

Human evaluation answers different questions from automated benchmarks. Preference ratings can capture usefulness, clarity, or style, but raters can prefer a persuasive wrong answer. Expert review can better assess specialized content but is slower and can still disagree. For high-stakes use, the appropriate test is often a task-specific evaluation with representative inputs, known ground truth, error analysis, and human oversight rather than a single general leaderboard score.

Limitations and risks

Hallucination and calibration

Hallucination refers to generated content that is unsupported or false despite sounding plausible. It follows partly from the next-token objective: the model is trained to continue text, not to consult a complete database and prove every statement. Post-training, retrieval, and tools can reduce some errors, but none makes arbitrary output automatically correct.^[5]

The GPT-4 report found better factuality than earlier GPT-3.5 versions on OpenAI's internal evaluations, yet continued to warn against uncritical use in high-stakes settings. It also noted that the post-trained model's confidence was not perfectly calibrated. Later launch posts report further reductions on selected internal evaluations, but such relative improvements do not establish a zero error rate.^{[5][21][24][28]}

Users should verify consequential claims against reliable sources. A citation generated by a model can be nonexistent, mismatched, or real but irrelevant. Search-enabled systems can still summarize a source incorrectly. Verification should inspect the cited material rather than treating the presence of a link as proof.

Prompt sensitivity and long-tail errors

Small changes in wording, examples, or context can change a response. Models can follow an incorrect premise, overlook a constraint, or solve a familiar-looking problem with an inappropriate pattern. Longer reasoning can catch some mistakes but can also elaborate an incorrect assumption. Reliability usually falls on rare, ambiguous, adversarial, or distribution-shifted inputs.

Long context creates additional failure modes. A model may retrieve one detail but fail to integrate several distant facts, be distracted by irrelevant passages, or give undue weight to text that resembles an instruction. Context-window size should therefore be reported separately from measured long-context accuracy.

Bias and representation

Web pages, books, code, and licensed corpora contain social stereotypes, factual errors, imbalanced language coverage, and uneven representation. Models can reproduce or amplify those patterns. Post-training can suppress some outputs but may also create uneven refusals or make behavior depend strongly on phrasing. Evaluation should cover different groups, languages, dialects, and use contexts rather than assuming one aggregate score represents everyone.

The GPT-1 release already acknowledged that text does not provide complete or unbiased knowledge of the world. InstructGPT later reported reduced toxic output under some prompting conditions but no significant improvement over GPT-3 on two bias datasets used in that study.^[3][13] Those results show why toxicity, stereotype, fairness, and representational harms should be measured separately.

Memorization, privacy, and copyright

Language models can memorize rare sequences from training. Carlini and colleagues used black-box queries to extract hundreds of verbatim GPT-2 training examples, including public personal information, even when some examples appeared in only one training document. In their tested attack, more than 600 of 1,800 high-ranked candidates were confirmed as verbatim training examples. The result demonstrates a real extraction risk, but it does not mean every generated phrase was memorized or that the same attack rate applies unchanged to every later model.^[30]

Deduplication, data filtering, access controls, monitoring, and privacy-preserving methods can reduce risk. They cannot retroactively establish that an undisclosed training corpus contains no personal or copyrighted material. Legal questions about training and output can also vary by jurisdiction and case. A factual article should distinguish a filed lawsuit, a judicial decision, a licensing agreement, and a research finding instead of grouping them as one settled conclusion.

Opacity and reproducibility

OpenAI published substantial architecture and data details for GPT-1 through GPT-3. GPT-4 marked a major reduction in technical disclosure. Without model size, training compute, or a reproducible data description, outside researchers cannot independently reconstruct the model or verify many claims about efficiency and scaling.^[5]

System cards and launch evaluations provide useful information about tested risks and mitigations, but they are produced by the developer and are not substitutes for independent access and replication. Closed weights and changing hosted snapshots also make it harder to reproduce an old result unless the exact model identifier, date, parameters, tools, and prompt are recorded.

Misuse and system security

GPT systems can lower the cost of producing spam, impersonation, deceptive content, insecure code, or instructions for harmful activity. GPT-2's staged release was an early attempt to study such concerns.^[9][10] Modern tool-using systems add risks such as prompt injection, data exfiltration, unauthorized actions, and accidental modification of external state.^[40][41]

Safety training can refuse or redirect some requests, but determined attackers may seek jailbreaks. Application security should use least-privilege permissions, sandboxing, input and output validation, audit logs, rate limits, and human confirmation for consequential actions. The surrounding product must enforce these controls; a natural-language instruction asking the model to be safe is not a complete security boundary.^[40][41]

Economic and social impact

ChatGPT brought GPT systems to a much wider public than the earlier research API, and GPT-based services are now used for writing, customer support, software development, education, document analysis, and information retrieval. Adoption alone does not measure productivity or welfare. Effects depend on the job, worker experience, interface, incentives, error cost, and whether a human can recognize bad suggestions.

One field study by Brynjolfsson, Li, and Raymond examined a staggered rollout of a GPT-based conversational assistant to 5,179 customer-support agents. The system showed suggestions to agents, who retained discretion over whether to use them. The revised working paper reported a 14 percent average increase in issues resolved per hour, a 34 percent improvement for novice and lower-skilled workers, and minimal average impact for the most experienced and highest-skilled workers. It also found evidence of improved customer sentiment and employee retention.^[35]

That study is evidence for one customer-service deployment, not proof that GPT increases output by the same amount in other occupations. It also studied an assistant combined with additional machine-learning components and company data. General claims about labor displacement, job creation, or economy-wide productivity require broader evidence than one model benchmark or one workplace rollout.

Educational and professional use raises a similar distinction between assistance and substitution. A model can provide explanations, drafts, and feedback, but it can also produce plausible errors or conceal gaps in a user's understanding. Institutions have responded with different rules for disclosure, citation, assessment, and confidential data. The appropriate policy depends on the purpose of the work and the consequences of error.

Other models using GPT in their names

GPT naming extends beyond OpenAI, but the examples are not one uniform lineage:

• GPT-J is a 6 billion parameter autoregressive model released by EleutherAI in 2021. Its design and weights were published independently of OpenAI.^[36]
• GPT-NeoX-20B is a 20 billion parameter autoregressive model and an open-source system for training large language models.^[37]
• BloombergGPT is a 50 billion parameter model developed for financial natural-language tasks using a mixture of financial and general-purpose data.^[38]
• BioGPT is a 347 million parameter generative Transformer developed for biomedical text generation and mining.^[39]

These examples show that GPT is used in research names for models with different developers, datasets, licenses, and target domains. They do not establish that Baidu's ERNIE family, Meta's Llama family, Anthropic's Claude family, or every autoregressive Transformer is a GPT. Architecture, training objective, branding, and model lineage should be described separately.

See also

• Generative AI
• Large Language Model
• Transformers
• OpenAI
• GPT-3
• GPT-4
• GPT-5
• ChatGPT
• In-Context Learning
• Reinforcement Learning from Human Feedback
• Scaling Laws

References

1. ^Vaswani, A. et al. (2017). "Attention Is All You Need." *Advances in Neural Information Processing Systems 30*. arxiv.org/...1706.03762
2. ^Radford, A.; Narasimhan, K.; Salimans, T.; Sutskever, I. (2018). "Improving Language Understanding by Generative Pre-Training." OpenAI. cdn.openai.com/...language_understanding_paper.pdf
3. ^OpenAI (June 11, 2018). "Improving language understanding with unsupervised learning." openai.com/...language-unsupervised
4. ^Radford, A. et al. (2019). "Language Models are Unsupervised Multitask Learners." OpenAI. cdn.openai.com/...language-models.pdf
5. ^OpenAI (2023). "GPT-4 Technical Report." arXiv:2303.08774. arxiv.org/...2303.08774
6. ^OpenAI (November 6, 2023). "Introducing GPTs." openai.com/...introducing-gpts
7. ^OpenAI (January 10, 2024). "Introducing the GPT Store." openai.com/...introducing-the-gpt-store
8. ^United States Patent and Trademark Office, Trademark Trial and Appeal Board. "Ex parte appeal, serial number 97733259, mark: GPT." Record current through June 13, 2026. ttabvue.uspto.gov/ttabvue
9. ^OpenAI (February 14, 2019). "Better language models and their implications." openai.com/...better-language-models
10. ^OpenAI (November 5, 2019). "GPT-2: 1.5B release." openai.com/...gpt-2-1-5b-release
11. ^Brown, T. B. et al. (2020). "Language Models are Few-Shot Learners." arXiv:2005.14165. arxiv.org/...2005.14165
12. ^OpenAI (June 11, 2020). "OpenAI API." openai.com/...openai-api
13. ^Ouyang, L. et al. (2022). "Training language models to follow instructions with human feedback." arXiv:2203.02155. arxiv.org/...2203.02155
14. ^OpenAI (November 30, 2022). "Introducing ChatGPT." openai.com/...chatgpt
15. ^OpenAI (March 14, 2023). "GPT-4." openai.com/...gpt-4-research
16. ^OpenAI (November 6, 2023). "New models and developer products announced at DevDay." openai.com/...veloper-products-announced-at-devday
17. ^OpenAI (May 13, 2024). "Hello GPT-4o." openai.com/...hello-gpt-4o
18. ^OpenAI (August 8, 2024). "GPT-4o System Card." openai.com/...gpt-4o-system-card
19. ^OpenAI (April 14, 2025). "Introducing GPT-4.1 in the API." openai.com/...gpt-4-1
20. ^OpenAI (September 12, 2024). "Learning to reason with LLMs." openai.com/...learning-to-reason-with-llms
21. ^OpenAI (August 7, 2025). "Introducing GPT-5." openai.com/...introducing-gpt-5
22. ^OpenAI (August 7, 2025). "Introducing GPT-5 for developers." openai.com/...introducing-gpt-5-for-developers
23. ^OpenAI (November 13, 2025). "Introducing GPT-5.1 for developers." openai.com/...gpt-5-1-for-developers
24. ^OpenAI (December 11, 2025). "Introducing GPT-5.2." openai.com/...introducing-gpt-5-2
25. ^OpenAI (March 3, 2026). "GPT-5.3 Instant: Smoother, more useful everyday conversations." openai.com/...gpt-5-3-instant
26. ^OpenAI (March 5, 2026). "Introducing GPT-5.4." openai.com/...introducing-gpt-5-4
27. ^OpenAI (April 23, 2026). "Introducing GPT-5.5." openai.com/...introducing-gpt-5-5
28. ^OpenAI (May 5, 2026). "GPT-5.5 Instant: smarter, clearer, and more personalized." openai.com/...gpt-5-5-instant
29. ^OpenAI (January 29, 2026). "Retiring GPT-4o, GPT-4.1, GPT-4.1 mini, and OpenAI o4-mini in ChatGPT." openai.com/...retiring-gpt-4o-and-older-models
30. ^Carlini, N. et al. (2021). "Extracting Training Data from Large Language Models." *30th USENIX Security Symposium*. arxiv.org/...2012.07805
31. ^Wei, J. et al. (2022). "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models." *Advances in Neural Information Processing Systems 35*. arxiv.org/...2201.11903
32. ^Kaplan, J. et al. (2020). "Scaling Laws for Neural Language Models." arXiv:2001.08361. arxiv.org/...2001.08361
33. ^Hoffmann, J. et al. (2022). "Training Compute-Optimal Large Language Models." arXiv:2203.15556. arxiv.org/...2203.15556
34. ^OpenAI (February 27, 2026). "An update on our mental health-related work." openai.com/...update-on-mental-health-related-work
35. ^Brynjolfsson, E.; Li, D.; Raymond, L. R. (2023, revised November 2023). "Generative AI at Work." NBER Working Paper 31161. nber.org/...w31161
36. ^Wang, B.; Komatsuzaki, A. (2021). "GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model." EleutherAI. github.com/...mesh-transformer-jax
37. ^Black, S. et al. (2022). "GPT-NeoX-20B: An Open-Source Autoregressive Language Model." arXiv:2204.06745. arxiv.org/...2204.06745
38. ^Wu, S. et al. (2023). "BloombergGPT: A Large Language Model for Finance." arXiv:2303.17564. arxiv.org/...2303.17564
39. ^Luo, R. et al. (2022). "BioGPT: generative pre-trained transformer for biomedical text generation and mining." *Briefings in Bioinformatics*, 23(6), bbac409. doi.org/...bbac409
40. ^OWASP Gen AI Security Project (2025). "LLM01:2025 Prompt Injection." genai.owasp.org/...llm01-prompt-injection
41. ^OWASP Gen AI Security Project (2025). "LLM06:2025 Excessive Agency." genai.owasp.org/...llm062025-excessive-agency
42. ^OpenAI (July 9, 2026). "GPT-5.6: Frontier intelligence that scales with your ambition." openai.com/...gpt-5-6