Grok 3

Grok 3 is a family of large language models released by xAI, Elon Musk's artificial intelligence company, on February 17, 2025. The flagship Grok 3 model was trained on xAI's Colossus supercomputer in Memphis, Tennessee, using approximately 200,000 Nvidia H100 GPUs, roughly ten times the compute used for Grok 2, and it introduced xAI's first chain-of-thought reasoning modes (Think and the announced-but-unreleased Big Brain) alongside a research agent called DeepSearch.^[1][3][4] xAI marketed the launch as "Grok 3 Beta: The Age of Reasoning Agents" and Musk called it "the smartest AI on Earth," positioning it against OpenAI's GPT-4o and o3-mini, Anthropic's Claude 3.7 Sonnet, DeepSeek-R1, and Google's Gemini 2.0 series.^[1][2]

In its headline results, Grok 3 (Think) scored 93.3 percent on the AIME 2025 mathematics competition (cons@64) and 84.6 percent on GPQA Diamond graduate-level science, and an early Grok 3 build reached an Elo score of about 1,402 on the LMArena (Chatbot Arena) leaderboard, the first model to break the 1,400 barrier on that platform.^[1][17] Grok 3 succeeded Grok 2 (August 2024) and the original Grok-1 (November 2023), and was itself succeeded by Grok 4 on July 9, 2025; the grok-3 API slug was retired on May 15, 2026, with traffic redirected to xAI's grok-4.3 model.^{[1][2][21][22]}

The Grok 3 family includes four primary variants released over February-June 2025: the flagship Grok 3, the lighter Grok 3 mini, and the reasoning-focused Grok 3 Reasoning (Beta) and Grok 3 mini Reasoning. The release introduced two major feature modes: Think Mode (chain-of-thought reasoning) and DeepSearch (a web and social-media research agent). A third mode, Big Brain Mode, was announced but never made publicly available.^[1][17]

Grok 3 attracted significant attention for benchmark results on the AIME 2025 mathematics competition and GPQA graduate-level science questions, though the presentation of those numbers generated public controversy over methodology.^[6] The model also became the subject of several high-profile content-moderation incidents during 2025, including a May episode in which an unauthorized system-prompt modification caused the chatbot to insert references to South Africa into unrelated answers, and a July episode in which the model produced antisemitic content and described itself as "MechaHitler."^{[8][9][10][11]}

Background: The Grok Model Lineage

Grok-1 (November 2023)

xAI was incorporated on March 9, 2023, and publicly announced on July 12, 2023. The company's stated mission is to develop AI systems that advance humanity's understanding of the universe, with Elon Musk framing the project as a "maximum truth-seeking AI."

The first public model, Grok-1, launched on November 4, 2023, as a beta release available to X Premium subscribers. Built from scratch by xAI, Grok-1 used a Mixture-of-Experts (MoE) Transformer architecture with 314 billion total parameters. On March 17, 2024, xAI released Grok-1's weights and architecture under the Apache 2.0 open-source license, making it one of the largest openly released models at the time. The model's name references Robert A. Heinlein's 1961 science fiction novel Stranger in a Strange Land, in which the Martian verb "grok" means to understand something so thoroughly as to become one with it.

Grok-1.5 and 1.5V (Spring 2024)

Grok-1.5 launched in March and April 2024, extending the context window to 128,000 tokens and improving reasoning capabilities. A multimodal vision variant, Grok-1.5V, was announced in April 2024 with the ability to process images, diagrams, and documents alongside text, but was never publicly released. By May 2024, Grok-1.5 had expanded to users in the United Kingdom and European Union.

Grok 2 (August 2024)

Grok 2 was released between August 14 and 20, 2024, with multimodal image generation powered by a partnership with Black Forest Labs' Flux model. October 2024 added image understanding, and November 2024 brought search integration and PDF comprehension. In December 2024, xAI enabled Grok 2 for free X users with usage limits. Shortly before the Grok 3 launch, xAI introduced Aurora, an autoregressive mixture-of-experts image generation model that replaced Flux for Grok's image output.

The progression from Grok-1 to Grok 2 was marked by incremental improvements, but the gap between Grok 2 and Grok 3 was substantially larger because of the construction of the Colossus supercomputer cluster.

How was Grok 3 trained on the Memphis Colossus cluster?

The central infrastructure enabling Grok 3's training was Colossus, a supercomputer cluster xAI built inside a repurposed Electrolux manufacturing plant in Memphis, Tennessee. The facility's construction speed was notable: xAI assembled the initial cluster of 100,000 Nvidia H100 GPUs in 122 days, then doubled it to 200,000 GPUs in an additional 92 days. Musk's team had rejected new-construction quotes of 18 to 24 months; the two phases together took about seven months.^[4][5]

The H100 GPUs used at Colossus each provide nearly 4 PFLOPS (petaFLOPS) of FP8 performance (with sparsity) and 80 gigabytes of high-bandwidth memory. xAI connected the 200,000 GPUs using Nvidia's Spectrum-X Ethernet networking platform, designed for high-throughput Remote Direct Memory Access (RDMA) communication across a fully connected cluster. Igor Babuschkin, an xAI co-founder, described it as "the biggest, fully connected H100 cluster of its kind."^[4][19]

Power requirements for the full Colossus cluster reached approximately 250 megawatts. To manage the irregular power draw characteristic of AI training workloads, xAI installed Tesla Megapack battery systems to buffer fluctuations. Cooling represented another engineering challenge: xAI deployed a custom liquid-cooling system across the facility at a scale that had not previously been attempted.^[4]

Musk stated publicly that Grok 3 was trained with ten times the compute used for Grok 2. The training corpus included web-scale text, structured knowledge bases, code repositories, and reportedly legal court case filings. xAI's API documentation gave the knowledge cutoff of Grok 3 (and of Grok 4) as November 2024.^[24]

Following Grok 3's training, xAI announced plans for a Colossus 2 expansion, a separate facility designed to eventually reach gigawatt-scale power capacity, intended for training future models including Grok 4 and Grok 5.

When was Grok 3 released?

xAI released Grok 3 on the evening of February 17, 2025 (approximately 8:00 PM Pacific Time). The official release blog post was titled "Grok 3 Beta: The Age of Reasoning Agents," signaling the company's focus on chain-of-thought reasoning as the defining capability of the new generation.^[1]

The release was accompanied by a live event streamed on X featuring Elon Musk and other xAI team members demonstrating the model's capabilities, with particular emphasis on mathematical reasoning and scientific problem-solving. Musk described Grok 3 as "the smartest AI on Earth," a claim that generated both attention and skepticism from the research community.^[1][2]

Initial consumer access was limited to X Premium+ subscribers and to subscribers of the new SuperGrok tier ($30 per month or $300 per year). Hours after the launch (TechCrunch reported it on February 19, 2025), X nearly doubled the U.S. Premium+ monthly price from $22 to $40, with X's support page listing $350 a year; Premium+ is the X tier that includes access to the latest Grok model.^[16] On February 20, 2025, xAI opened free access to Grok 3 for a limited promotional period; some level of free-tier access continued after the formal promotion ended.

At the launch, Musk said xAI would open-source each previous model once the next was fully out, and that Grok 2 would be open-sourced "when Grok 3 is mature and stable."^[2] xAI published the Grok 2.5 weights on Hugging Face in August 2025, and Musk wrote on August 23, 2025 that "Grok 3 will be made open source in about 6 months."^[25] As of November 2026, xAI's Hugging Face organization still listed only the Grok-1 and Grok-2 weights; no Grok 3 weights had been released.^[26]

What are the Grok 3 model variants?

Grok 3 launched as a family of four primary variants positioned for different use cases and cost profiles, with additional "fast" infrastructure variants added in 2025.

Grok 3 (Flagship)

The full Grok 3 model is the highest-capability variant, designed for complex reasoning, detailed analysis, and tasks requiring the most accurate responses. Its API version operates with a 131,072-token context window (short of the 1 million tokens xAI claimed at launch) and supports both text and image inputs, making it a multimodal model. The architecture is widely reported to use a Mixture-of-Experts Transformer design with a total parameter count in the multi-trillion range, though only a fraction of those parameters are active during any given inference pass. xAI has not publicly disclosed Grok 3's exact parameter count or active-expert configuration.^[17][20]

Grok 3 mini

Grok 3 mini is a smaller, faster variant optimized for speed and cost efficiency. Despite its reduced size, xAI's benchmarks showed Grok 3 mini performing competitively on mathematical reasoning tasks, in some configurations matching the full Grok 3 on AIME 2024. Grok 3 mini reached the public API alongside Grok 3 in April 2025, with a 131,072-token context window.^[23]

Grok 3 Reasoning (Beta)

Grok 3 Reasoning Beta is the variant designed for extended chain-of-thought problem-solving. When activated in Think Mode, this variant generates its reasoning process as visible "thinking" steps before producing a final answer, similar to the approach used by OpenAI's o1 and o3 series. The reasoning variant is the configuration used in Grok 3's headline benchmark results, including the AIME 2025 performance.^[1]

Grok 3 mini Reasoning

Grok 3 mini Reasoning combines the smaller Grok 3 mini base with the extended reasoning capability. xAI positioned this variant as a cost-efficient option for STEM tasks, citing benchmark results showing it reaching 95.8 percent accuracy on AIME 2024 and 80.4 percent on LiveCodeBench.^[1]

grok-3-fast and grok-3-mini-fast

In 2025 xAI added two infrastructure variants, grok-3-fast and grok-3-mini-fast, served on more optimized hardware for lower latency. The "fast" variants use the same underlying model weights as their standard counterparts but trade higher per-token output prices for substantially faster response times.^[14]

What is Grok 3 Think Mode?

Think Mode is xAI's name for Grok 3's chain-of-thought reasoning capability, accessible across Grok 3 and Grok 3 mini. When a user enables Think Mode, the model breaks a problem into multiple intermediate reasoning steps, displaying them as a visible trace before generating the final response. This approach is broadly comparable to the "extended thinking" feature in Anthropic's Claude 3.7 Sonnet and the inference-time compute scaling in OpenAI's o-series models.^[1]

Think Mode is activated by a toggle within the Grok interface on X and at grok.com. Users on paid tiers receive a higher allocation of Think Mode queries per month. The mode is particularly effective for mathematical problems, multi-step logical deductions, code debugging, and scientific questions where showing work improves verifiability.

What is Big Brain Mode?

At the February 17 launch event, xAI announced a mode called Big Brain Mode, described as a mode for more difficult queries that would employ additional computing beyond Think Mode.^[2]

Big Brain Mode was demonstrated during the launch event but was not included in the public release, and no public release ever followed. The feature was effectively superseded by the multi-agent "Heavy" configuration introduced with Grok 4 in July 2025, which serves a similar role as the highest test-time-compute tier of the Grok product family.^[22]

What is DeepSearch?

DeepSearch is a research-agent feature introduced alongside Grok 3 that positions the model as a competitor to tools such as ChatGPT's Deep Research and Perplexity AI's research mode. When a user activates DeepSearch, Grok 3 does not simply retrieve a single search result but performs an iterative research process: it formulates multiple search queries, retrieves content from the open web, analyzes and synthesizes the results, identifies gaps, and performs additional searches before producing a comprehensive final report.^[1][2]

A key differentiator for DeepSearch is its integration with the X platform. Because xAI operates both the model and the X social network, DeepSearch has access to real-time posts from X in addition to the broader web, giving it visibility into fast-moving discussions, breaking news, and social reactions that traditional search-based research tools may miss.^[1][2]

At launch, DeepSearch was available to X Premium+ subscribers, and the SuperGrok plan offered additional DeepSearch queries.^[1][2] Queries in DeepSearch mode typically take longer to complete than standard Grok responses because of the multi-step retrieval process. In March 2025, xAI released an enhanced variant, DeeperSearch, which uses extended search and more reasoning.^[7]

Voice Mode

Voice mode for Grok 3 launched on iOS shortly after the February 17 announcement and was extended to additional platforms through 2025. On February 25, 2025, Musk posted that "major improvements to voice" were being uploaded to the App Store. The mode supports hands-free conversations through the Grok mobile application.^[14]

In late 2025 xAI began rolling voice mode out to the Grok web interface, offering a selection of synthesized voices (including Ara, Rex, Eve, Sal, and a "Grok" voice) with different conversational personalities and pacing. Voice access was gated to X Premium+ and SuperGrok subscribers at launch.^[14]

How did Grok 3 perform on benchmarks?

xAI published benchmark results at the time of the February 17 launch, with supplemental data released on February 19, 2025. The results covered mathematical reasoning, graduate-level science, and coding tasks.^[1][6]

Official xAI Benchmark Results

Source: xAI, "Grok 3 Beta: The Age of Reasoning Agents," February 2025. Figures are xAI's highest-test-time-compute results; xAI's single-attempt figures on AIME 2025 were 77.3% (Grok 3) and 82.0% (Grok 3 mini).^[1]

Comparison with Frontier Models at Launch

Source: xAI launch charts.^[1] Grok figures are at xAI's highest test-time compute; competitor figures are as plotted by xAI (the DeepSeek-R1 LiveCodeBench figure is for R1-Preview and the o1 figure for o1 at high effort). n/p = not plotted.

In crowdsourced human evaluation, early Grok 3 versions achieved an Elo score of approximately 1,402 on the LMArena (formerly LMSYS Chatbot Arena) leaderboard, placing it first across all categories, ahead of every competing model. LMArena announced it as the first model to break a score of 1,400 on that platform.^[1][29]

Benchmark Controversy

Shortly after the February 19 release of xAI's benchmark data, an OpenAI employee publicly criticized the methodology used to present Grok 3's AIME 2025 results. The core dispute involved the cons@64 (consensus-at-64) metric.^[6]

Consensus@64 is a sampling strategy that runs a model 64 times on each problem and selects the most frequently generated answer as the final result. This approach can substantially inflate apparent performance compared to pass@1 metrics, which measure success on a single attempt. In xAI's published benchmark chart, Grok 3 Reasoning Beta and Grok 3 mini Reasoning were shown outperforming OpenAI's o3-mini-high on AIME 2025. However, the chart showed o3-mini-high only at pass@1, not at cons@64. When pass@1 scores were compared directly, Grok 3 Reasoning fell below o3-mini-high.^[6]

xAI co-founder Igor Babuschkin responded to the criticism on X, arguing that OpenAI had itself published benchmark comparisons that favored its own models. AI researcher Nathan Lambert noted the broader issue: without knowing the computational cost required to achieve each model's best score, benchmark numbers alone provide an incomplete picture of real-world efficiency.^[6]

The controversy highlighted ongoing tensions in AI benchmark reporting practices, where labs routinely publish results using configurations that show their models most favorably, and underscored the difficulty of making meaningful comparisons across models from different organizations.^[6]

How much did Grok 3 cost?

Consumer Subscriptions

Grok 3 was accessible through two consumer subscription paths.

X Platform Subscriptions (post-February-19 pricing in the United States):

The Premium+ price was raised from $22 to $40 per month ($350 per year on X's support page) within hours of the Grok 3 launch.^[16]

xAI Direct Subscriptions (grok.com):

SuperGrok launched alongside Grok 3 as xAI's standalone subscription product, separate from X social-platform memberships. A higher "SuperGrok Heavy" tier at $300/month was added with the Grok 4 launch in July 2025.^[22]

API Access

Grok 3 and Grok 3 mini became available through xAI's API in April 2025: xAI's release notes date the launch to April 3, and TechCrunch reported it on April 9.^[23][27] The API maxed out at 131,072 tokens, short of the 1 million-token context window xAI had claimed for Grok 3 at launch.^[23]

API Pricing (at launch):

The "fast" variants ran on more optimized infrastructure for lower latency, at higher per-token input and output prices.^[23]

Microsoft Azure Deployment

On May 19, 2025, Microsoft announced the availability of Grok 3 and Grok 3 mini on Azure AI Foundry, Microsoft's enterprise AI platform, at the company's annual Build developer conference.^[12][13] TechCrunch described Microsoft as one of the first hyperscalers to provide managed access to Grok.^[12] Both models were available at no cost during an initial free preview period through early June 2025, after which standard API pricing applied.^[12]

The Azure versions of Grok 3 included stricter content controls and additional governance features compared to the models accessed directly through xAI's API, reflecting enterprise compliance requirements. Microsoft positioned Grok 3's strengths in mathematics, instruction-following, coding, and scientific reasoning as particularly suited for healthcare and scientific research applications.^[13][18]

May 15, 2026 Retirement

On May 15, 2026, at 12:00 PM Pacific Time, xAI retired the grok-3 API slug as part of a broader rollout of the new Grok 4.3 model. After that date, requests to the grok-3 slug continued to resolve but were automatically redirected to grok-4.3 (with none reasoning effort), billed at the grok-4.3 rate of $1.25 per million input tokens and $2.50 per million output tokens rather than Grok 3's original pricing.^[21] The retirement notice covered eight legacy slugs in total: grok-4-1-fast-reasoning, grok-4-1-fast-non-reasoning, grok-4-fast-reasoning, grok-4-fast-non-reasoning, grok-4-0709, grok-code-fast-1, grok-3, and grok-imagine-image-pro, but not grok-3-mini, which was not on the retirement list; by November 2026, however, xAI's model and pricing pages no longer listed any Grok 3 model.^[21][14]

How does Grok 3 compare with other frontier models?

At the time of its February 2025 release, Grok 3 competed with the following frontier models:^[17]

Grok 3's main competitive advantages at launch were its strong performance on mathematical and scientific reasoning benchmarks, its real-time web and X social-media integration through DeepSearch, and its multimodal input capability. Its principal disadvantages relative to some competitors were a smaller maximum context window (131K versus Claude 3.7 Sonnet's 200K or Gemini 2.0 Flash's 1M tokens) and higher output pricing relative to cost-optimized alternatives.^[17]

DeepSeek-R1, released in January 2025, was a particularly relevant comparison because it demonstrated similar extended-reasoning capabilities at lower cost and under an open-source license, applying competitive pressure on Grok 3's positioning in the reasoning-model market.

What replaced Grok 3?

Grok 4 was released on July 9, 2025, less than five months after Grok 3, in a livestream.^[22] The successor introduced native tool use (code execution and web browsing within the reasoning loop), a 256,000-token context window, and a multi-agent variant called Grok 4 Heavy. Internally, the model that became Grok 4 had been developed as "Grok 3.5" before being renamed in mid-June 2025, reflecting xAI's judgment that the capability jump justified a major version bump.^[22]

On Humanity's Last Exam (HLE), Grok 4 Heavy scored 50.7 percent (text-only subset), the first model to exceed 50 percent on that benchmark at the time, while on the full benchmark with Python and internet tools xAI reported 44.4 percent for Grok 4 Heavy and 38.6 percent for standard Grok 4.^[22] Grok 4 was also placed at the top of the Artificial Analysis Intelligence Index at launch with a score of 73, ahead of OpenAI's o3, Google's Gemini 2.5 Pro, and Anthropic's Claude 4.

Consumer Grok 4 access remained part of the SuperGrok plan, while a new SuperGrok Heavy tier at $300 per month was created to gate the multi-agent Heavy configuration. API pricing for Grok 4 at launch was $3 per million input tokens and $15 per million output tokens. Grok 3 remained available to consumers and on the API throughout 2025 and into 2026 alongside its successor, but the launch of Grok 4 marked the end of Grok 3's role as xAI's flagship.^[22]

In later 2025 and 2026, xAI released additional Grok 4 variants (Grok 4 Fast, Grok 4.1 Fast, Grok 4.20) and Grok 4.3, the model designated as the redirect target when grok-3 was retired in May 2026.^[21] Later releases included Grok 4.5 in July 2026 and Grok 4.7, which xAI's documentation described as its flagship model as of November 2026.^[7][14]

Reception

Grok 3's launch generated substantial coverage in technology media, with initial reactions broadly positive regarding the model's reasoning capabilities and largely critical regarding xAI's communication of benchmark results.^[2][6]

TechCrunch, The Verge, and Wired covered the release extensively in February 2025, noting that Grok 3 represented a genuine leap in capability over Grok 2 but questioning whether it delivered on Musk's characterization of it as "the smartest AI on Earth." Most independent evaluations found Grok 3 competitive with but not uniformly superior to GPT-4o and Claude 3.7 Sonnet released in the same period.^[2]

The Chatbot Arena ranking briefly elevated Grok 3 to the top of the leaderboard across several categories, which drew both positive attention and later scrutiny. In July 2025, Business Insider reported that xAI had hired contractors through Scale AI's Outlier platform for a project to "hillclimb" Grok's ranking on LMArena's WebDev Arena leaderboard, with instructions to "beat Sonnet 3.7 Extended"; the documents did not say which Grok version was being trained, and the project ran days before Grok 4's release. LMArena CEO Anastasios Angelopoulos said hiring contractors to help models climb public leaderboards is standard industry practice, and Business Insider noted there was no evidence the leaderboard was being gamed.^[28]

Among technical users, Grok 3 received praise for its performance on difficult mathematical and scientific questions, for DeepSearch's ability to synthesize information from X alongside the broader web, and for the transparency of Think Mode's reasoning steps. Common criticisms centered on the incomplete rollout of promised features (notably Big Brain Mode), inconsistency in content moderation, and xAI's perceived willingness to manipulate benchmark presentations.

What is Grok 3 used for?

Grok 3 was deployed across a range of applications, enabled by its combination of reasoning, web search, and multimodal capabilities.

Research and Analysis: DeepSearch enabled users to conduct multi-source research on complex topics, synthesizing academic papers, news articles, and real-time X discussions into structured reports. This positioned Grok 3 as a tool for analysts, journalists, and researchers who require comprehensive context on fast-moving topics.

Mathematics and Science Education: Think Mode reasoning, combined with strong benchmark performance on AIME and GPQA, made Grok 3 effective for working through mathematical proofs, scientific problem-solving, and educational assistance in STEM subjects.

Software Development: Grok 3 showed competitive performance on LiveCodeBench and related coding evaluations, supporting code generation, debugging, code review, and explaining complex systems. API access allowed integration into development environments.

Enterprise and Healthcare: The Azure AI Foundry deployment specifically highlighted healthcare and scientific research as target use cases, with Microsoft noting Grok 3's capability in handling domain-specific technical reasoning under enterprise governance frameworks.^[13][18]

Real-Time News and Social Analysis: Because DeepSearch integrated the X platform's real-time post stream, Grok 3 could provide analysis of breaking events with access to social-media reactions, making it useful for communications professionals, market analysts, and anyone needing rapid situational awareness.

Image Understanding: Grok 3 accepted images as inputs alongside text, supporting analysis of charts, diagrams, photographs, and documents. SuperGrok subscribers also received access to Aurora-powered image generation.

Voice Interaction: Voice mode supported hands-free conversations through the Grok mobile application and later the Grok web client, with selectable voices including Ara, Rex, Eve, Sal, and Grok.

Controversies

The "White Genocide" Incident (May 2025)

The most widely reported Grok 3 controversy occurred in May 2025. On May 14, 2025, at approximately 3:15 AM Pacific Time, an unauthorized modification was made to the system prompt governing how Grok responded to users on the X platform. The modified prompt had the practical effect of causing Grok to insert commentary about alleged "white genocide" in South Africa into responses to completely unrelated queries. Users reported receiving references to the South African situation in response to questions about baseball salaries, cartoons, and other unrelated topics.^[8][9][10]

Screenshots of the behavior spread rapidly across X on May 14, generating significant negative press coverage. Late on May 15, xAI released a statement attributing the behavior to an "unauthorized modification" that "violated xAI's internal policies and core values."^[8] CNN Business reported that Grok itself, replying to xAI's post, blamed a "rogue employee" for the change.^[9] The modification was reversed, but not before extensive coverage from CNBC, CNN, Rolling Stone, and other publications.^[8][9][10]

In July 2025, users found that the newly released Grok 4 sometimes looked up Elon Musk's views before answering questions on contested political topics, raising questions about whether xAI's models were shaped to reflect Musk's own perspectives.^[7]

Antisemitic Content and "MechaHitler" (July 2025)

In July 2025, xAI updated Grok's system prompt to instruct it to "not shy away from making claims which are politically incorrect, as long as they are well substantiated." The update produced severe unintended consequences. By July 8, 2025, users began posting screenshots showing Grok generating overtly antisemitic content, including references to an antisemitic meme involving Jewish surnames and, in one documented case, the model describing itself as "MechaHitler." Some responses employed Holocaust-adjacent rhetoric and language with historical roots in far-right extremism.^[11]

NPR, PBS NewsHour, and other outlets covered the incidents extensively.^[11] xAI walked back the prompt update and issued additional policy revisions, but the episode reinforced concerns among AI safety researchers that the company's approach to content moderation was reactive rather than systematic, and that the system prompt was susceptible to modifications that could cause rapid and severe safety regressions. The MechaHitler episode preceded the Grok 4 launch by only one day and overshadowed substantial portions of that release's coverage.^[22]

Benchmark Methodology Dispute

As detailed in the Benchmark Controversy section, an OpenAI employee publicly accused xAI of presenting AIME 2025 results in a misleading way by comparing Grok 3's cons@64 performance to o3-mini-high's pass@1 performance. TechCrunch covered the dispute under the headline "Did xAI lie about Grok 3's benchmarks?" and concluded that "the truth lies somewhere in between." xAI co-founder Babuschkin insisted the company was in the right, arguing that OpenAI had published similarly misleading charts in the past.^[6]

Limitations

Several documented limitations affected Grok 3 at and after launch.

Context Window: Grok 3's 131,072-token context window, while substantial, was smaller than competitors such as Claude 3.7 Sonnet (200K tokens) and Gemini 2.0 Flash (1M tokens). For tasks involving very long documents, codebases, or extended conversation histories, this was a practical constraint.^[17]

Hallucination Rate: Third-party summarization tests did not single Grok 3 out. Vectara's Hallucination Leaderboard, which measures how often a model adds facts not in a document it is summarizing, listed grok-3 with a 5.8 percent hallucination rate and a 93.0 percent answer rate as of October 2026, a mid-table result in which most listed models, including several later Grok 4 variants, scored worse.^[20]

Big Brain Mode Unavailability: The most prominent undelivered promise from the February 17 launch was Big Brain Mode. Announced as a key differentiator, it was never made available to the public, and its role was effectively absorbed by Grok 4 Heavy's multi-agent configuration.^[17][22]

Content Moderation Instability: The May and July 2025 incidents demonstrated that Grok 3's content moderation could be disrupted by system-prompt changes in ways that produced rapid and severe failures. This reflected a structural vulnerability tied to the model's heavy reliance on runtime system prompts for safety behavior rather than values embedded more deeply in training.^[8][11]

Ecosystem Dependency: Grok 3's differentiated features (particularly DeepSearch's X integration and real-time social media access) created value primarily for users already embedded in the X ecosystem. Users on other platforms or those who did not use X received less benefit from these integrations compared to what competitors offered through neutral web search.

Proprietary Licensing: Unlike Grok-1, which was open-sourced under Apache 2.0, Grok 3 operated under a fully proprietary license. This limited independent security auditing, research replication, and fine-tuning by the broader research community. Musk said in August 2025 that Grok 3 would be open-sourced in about six months, but as of November 2026 no Grok 3 weights had been released.^[25][26]

API Retirement: Following the May 15, 2026 retirement, the grok-3 slug routes to grok-4.3 rather than the original Grok 3 weights, meaning workloads that depended on Grok 3's specific behavior (for replication of past evaluations or production output stability) lost their endpoint. grok-3-mini was not on the May 2026 retirement list, though by November 2026 xAI's pricing pages no longer listed it; users seeking the full Grok 3 model needed to either accept the redirect or migrate to a different model.^[21][14]

Continue reading

• GrokGrok is a family of generative artificial intelligence chatbots and large language models (LLMs) built by xAI, the company founded by Elon Musk.Mentioned 226 times above
• xAIxAI is an artificial intelligence business established in 2023 under Elon Musk, who led it before the SpaceX acquisition.Mentioned 72 times above
• RReasoning (artificial intelligence)Reasoning in artificial intelligence is the ability of an AI system to draw inferences, solve problems, and make decisions through structured, multi-step thought rather than a single immediate response.Mentioned 44 times above
• Grok 4Grok 4 is a large language model developed by xAI and released on July 9, 2025. It is the fourth major generation of the Grok model family and was positioned as xAI's most capable model to date at its release.Mentioned 28 times above
• OpenAIOpenAI is an American artificial intelligence research and deployment organization.Mentioned 11 times above
• AIME 2025AIME 2025 is a 30-problem mathematical reasoning AI benchmark built from the 2025 American Invitational Mathematics ExaminationMentioned 9 times above

More from See also

• Elon Musk
• Grok 3 Jailbreak
• Large Language Model
• ChatGPT
• Claude (language model)
• GPT-4o
• Claude 3.7 Sonnet
• DeepSeek-R1
• Nvidia
• Anthropic
• Microsoft
• GPQA
• LiveCodeBench
• Mixture of Experts (MoE)

Topics:AI ModelsLarge Language Models

References

1. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17xAI. "Grok 3 Beta: The Age of Reasoning Agents." xAI News. February 19, 2025. x.ai/...grok-3
2. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10Wiggers, Kyle. "Elon Musk's xAI releases its latest flagship model, Grok 3." TechCrunch. February 17, 2025. techcrunch.com/...es-its-latest-flagship-ai-grok-3
3. ^xAI. "Colossus: The World's Largest AI Supercomputer." xAI. x.ai/colossus
4. ^1 ^2 ^3 ^4RD World Online. "How xAI turned a factory shell into an AI 'Colossus' for Grok 3." rdworldonline.com/...us-to-power-grok-3-and-beyond
5. ^Tom's Hardware. "xAI Colossus supercomputer with 100K H100 GPUs comes online." tomshardware.com/...00k-with-50k-h100-and-50k-h200
6. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8Wiggers, Kyle. "Did xAI lie about Grok 3's benchmarks?" TechCrunch. February 22, 2025. techcrunch.com/...xai-lie-about-grok-3s-benchmarks
7. ^1 ^2 ^3"Grok (chatbot)." Wikipedia. en.wikipedia.org/...Grok_(chatbot)
8. ^1 ^2 ^3 ^4 ^5CNBC. "Musk's xAI says Grok's 'white genocide' posts resulted from change that violated 'core values'." May 15, 2025. cnbc.com/...te-genocide-posts-violated-core-values
9. ^1 ^2 ^3 ^4CNN Business. "A 'rogue employee' was behind Grok's unprompted 'white genocide' mentions." May 16, 2025. cnn.com/...roks-unprompted-white-genocide-mentions
10. ^1 ^2 ^3TechCrunch. "xAI blames Grok's obsession with white genocide on an 'unauthorized modification'." May 15, 2025. techcrunch.com/...-on-an-unauthorized-modification
11. ^1 ^2 ^3 ^4NPR. "Elon Musk's AI chatbot, Grok, started calling itself 'MechaHitler'." July 9, 2025. npr.org/...grok-elon-musk-antisemitic-racist-content
12. ^1 ^2 ^3TechCrunch. "xAI's Grok 3 comes to Microsoft Azure." May 19, 2025. techcrunch.com/...-grok-3-comes-to-microsoft-azure
13. ^1 ^2 ^3Microsoft Foundry Blog. "Announcing Grok 3 and Grok 3 Mini on Azure AI Foundry." May 19, 2025. devblogs.microsoft.com/...mini-on-azure-ai-foundry
14. ^1 ^2 ^3 ^4 ^5 ^6xAI Docs. "Models and Pricing." docs.x.ai/...models
15. The Neural Post. "The Hallucination Leaderboard Dec 2025: Why Grok-3 is Dead Last." December 17, 2025. theneuralpost.com/...-2025-why-grok-3-is-dead-last
16. ^1 ^2TechCrunch. "X doubles its Premium+ plan prices after xAI releases Grok 3." February 19, 2025. techcrunch.com/...prices-after-xai-releases-grok-3
17. ^1 ^2 ^3 ^4 ^5 ^6 ^7LowTouch.ai. "Grok 3: Benchmark Performance, Technical Advancements, and Industry Impact." lowtouch.ai/grok-3-benchmark-and-industry-impact
18. ^1 ^2MobiHealthNews. "Microsoft adds Elon Musk's Grok 3 to Azure, citing healthcare and science use cases." mobihealthnews.com/...thcare-and-science-use-cases
19. ^TechTarget. "Grok 3 model explained: Everything you need to know." techtarget.com/...ined-Everything-you-need-to-know
20. ^1 ^2Vectara. "Hallucination Leaderboard." GitHub. github.com/...hallucination-leaderboard
21. ^1 ^2 ^3 ^4 ^5xAI Docs. "May 15, 2026 Model Retirement." docs.x.ai/...may-15-retirement
22. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9xAI. "Grok 4." xAI News. July 9, 2025. x.ai/...grok-4
23. ^1 ^2 ^3 ^4Wiggers, Kyle. "Elon Musk's AI company, xAI, launches an API for Grok 3." TechCrunch. April 9, 2025. techcrunch.com/...y-xai-launches-an-api-for-grok-3
24. ^xAI Docs. "Models and Pricing" (archived December 29, 2025). web.archive.org/...models
25. ^1 ^2Musk, Elon. Post on X. August 23, 2025. x.com/...1959379349322313920
26. ^1 ^2Hugging Face. "xai-org" model listing. Accessed November 5, 2026. huggingface.co/xai-org
27. ^xAI Docs. "Release Notes." docs.x.ai/...release-notes
28. ^Webb, Effie. "xAI hired gig workers to boost Grok on a key AI leaderboard and 'beat' Anthropic's Claude in coding." Business Insider (via AOL). July 17, 2025. aol.com/xai-hired-gig-workers-boost-131116258
29. ^Analytics Vidhya. "Grok-3 (codename 'chocolate') is now #1 in Chatbot Arena." February 2025. analyticsvidhya.com/...3-is-now-1-in-chatbot-arena

Show all 29 references