Coreference Resolution

Coreference resolution is the natural language processing task of finding all the expressions in a text that refer to the same entity and grouping them together. The individual expressions are called mentions, and a group of mentions that all point at one entity is called a coreference chain, a cluster, or simply an entity ^[1]. A system given the passage "Victoria Chen, CFO of Megabucks Banking, saw her pay jump to $2.3 million, as the 38-year-old became the company's president" has to decide that Victoria Chen, her, and the 38-year-old belong in one cluster and that Megabucks Banking and the company belong in another. Practical systems have to do two jobs at once: detect which spans of text are mentions in the first place, and decide which of those mentions corefer ^[1].

The task is closely tied to, but not identical with, anaphora. Reference to an entity already introduced in the discourse is anaphora, and the referring expression is an anaphor; the earlier mention it refers back to is the antecedent. Pronouns can also point forward, which is called cataphora. An entity with only one mention in a document is a singleton ^[1]. Coreference is a symmetric relation between mentions of the same entity, whereas anaphora is a directional dependency on prior context, so the two overlap heavily without coinciding.

Coreference has been a standard evaluation task since 1995 and remains one of the most durable benchmarks in NLP. Scores on the main English benchmark climbed from roughly 60 CoNLL F1 in 2013 to 83.6 in 2024, driven first by neural network mention ranking, then by pretrained encoders such as BERT, and most recently by careful engineering of comparatively small discriminative models ^{[2][3][4][14]}. The task has also served a second role as a diagnostic: Winograd schemas, gender-bias probes, and long-document benchmarks all use pronoun resolution to test whether a model has learned something beyond surface word association.

Linguistic background

Not every noun phrase in a document is a mention. Appositives ("Richard Godown, president of the Industrial Biotechnology Association") are usually folded into the mention they describe rather than treated as separate entities. Predicative and attributive noun phrases describe properties rather than picking out a distinct referent, so "a unit of UAL" in "United is a unit of UAL" is not annotated as a mention. Pleonastic pronouns, as in "It is raining", refer to nothing at all ^[1].

Some cases are genuinely hard for reasons that have nothing to do with syntax. More than half of definite noun phrases in newswire text are non-anaphoric, typically because they introduce an entity for the first time ^[1]. Pro-drop languages such as Chinese and Japanese leave subjects unexpressed, so the mention that has to be resolved may not appear on the page at all. Related discourse phenomena, including bridging reference and discourse deixis, are annotated in specialist corpora such as ARRAU but excluded from the mainstream benchmark ^[1].

From MUC-6 to OntoNotes

Coreference became a formal shared-task evaluation at the Sixth Message Understanding Conference. A planning committee had proposed three semantic evaluations, coreference, word sense disambiguation, and predicate-argument structure, and at a June 1994 meeting chose coreference on the grounds that it was the most tractable of the three and the most critical to information extraction. The specification was narrowed to identity relations, dropping the set-subset and part-whole relations that the initial draft had envisioned. The formal evaluation was held in September 1995 with four tasks: named entity, coreference, template element, and scenario template. Of the 16 participants, 7 entered the coreference task, which was annotated in SGML using COREF tags carrying ID and REF attributes ^[5]. The same evaluation established named entity recognition as a task in its own right.

The benchmark that defined the modern era is OntoNotes, released in its fifth version by the Linguistic Data Consortium in October 2013. OntoNotes layers coreference annotation on top of Treebank syntax, PropBank predicate-argument structure, and word sense annotation across English, Chinese, and Arabic ^[6]. Its English and Chinese coreference portions run to roughly one million words each, with about 300,000 words of Arabic newswire. Its most consequential design decision is that it does not annotate singletons, which removes 60 to 70 percent of all entities from the problem and makes OntoNotes scores incomparable with corpora that do mark them ^[1].

Two CoNLL shared tasks turned OntoNotes into the field's standard leaderboard: CoNLL-2011, on English only, and CoNLL-2012, which extended the task to Chinese and Arabic ^[7][8]. The English split of 2,802 training, 343 development, and 348 test documents has been reused essentially unchanged for more than a decade ^[2]. Before neural methods took over, the strongest systems included the Stanford multi-pass sieve, a deterministic architecture that applies a cascade of precision-ranked rules to build entity clusters ^[9].

Model architectures

Systems are usually grouped by whether they reason about mentions or about entities, and by whether they rank candidate antecedents against each other ^[1].

The mention-pair architecture is the oldest and simplest. A binary classifier is given a candidate anaphor and a candidate antecedent and predicts whether they corefer; the transitive closure of the positive pairs forms the clusters. Because most mention pairs in a document are not coreferent, training uses a sampling heuristic, most commonly taking the closest true antecedent as the positive example and everything in between as negatives. The architecture has two structural weaknesses: it never compares two plausible antecedents against each other, and it makes every decision locally without any representation of the entity ^[1].

Mention-ranking models fix the first weakness. For each anaphor, the model computes a single distribution over all previous mentions plus a dummy antecedent that means "this mention starts a new chain", which lets anaphoricity detection and antecedent selection be learned jointly. Training is harder because the correct antecedent is latent: any member of the gold cluster is a legal choice, so the loss sums over all of them ^[1].

Entity-based models fix the second weakness by linking a mention to a cluster rather than to an earlier mention, using features such as cluster size and cluster shape or learned cluster representations. In practice the extra expressiveness has not translated into large gains, and mention ranking remains the dominant design ^[1].

Neural coreference resolution

The 2017 end-to-end model of Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer removed the pipeline entirely. It considers every span in the document as a possible mention, scores spans and antecedents jointly, and prunes aggressively rather than relying on a syntactic parser or a hand-built mention detector. Span embeddings combined LSTM boundary representations with a head-finding attention mechanism over the span, and word representations were a fixed concatenation of GloVe and Turian embeddings. A single model reached 67.2 average CoNLL F1 and a five-model ensemble 68.8, against 65.7 for the previous best system ^[2].

The 2018 follow-up added coarse-to-fine antecedent pruning and a differentiable approximation of higher-order inference, using the antecedent distribution as attention to refine span representations iteratively. Combined with ELMo contextual embeddings, it reached 73.0 ^[3]. Replacing ELMo with BERT-large lifted the same architecture to 76.9, and the span-aware pretraining of SpanBERT lifted it again to 79.6, at the time a new state of the art ^[4][10]. SpanBERT differs from BERT by masking contiguous spans instead of random tokens and training boundary representations to predict the whole masked span ^[10].

Later work attacked the quadratic and quartic blowup in spans and span pairs. Word-level coreference resolution links individual words and reconstructs spans afterwards, which cuts the model to quadratic complexity, removes the need for pruning, and reaches 81.0 with a RoBERTa encoder while cutting peak inference memory from 7.4 to 2.9 GiB against the same architecture run at the span level ^[11]. LingMess replaces the single pairwise scorer with six scorers, one for each of a linguistically motivated set of mention-pair categories ^[12].

Around 2022 the field briefly converged on large generative models. A transition-based sequence-to-sequence system built on multilingual T5 predicted mentions and links jointly as text, reaching 83.3 F1 on English CoNLL-2012, 68.5 on Arabic, and 74.3 on Chinese ^[13]. Maverick, presented at ACL 2024, pushed back on that trend: a carefully engineered DeBERTa-based discriminative pipeline with 504 million parameters reached 83.6 CoNLL F1, beating 11 to 13 billion parameter generative systems while training on a single consumer GPU, using as little as 0.006 times the memory of the previous state of the art and running inference about 170 times faster ^[14].

Scroll sideways for more →

Scores are average CoNLL F1 on the English CoNLL-2012 test set. The 2013 to 2017 rows are as compiled by Lee et al., the 2018 and 2019 rows come from the papers that introduced them, and the 2020 to 2024 rows are as compiled by Martinelli et al. ^{[2][3][4][14]}.

Evaluation metrics

Coreference is scored model-theoretically, by comparing the system's clusters against a human-annotated gold partition. Five metrics are in common use, and none of them is satisfactory alone ^[1].

Scroll sideways for more →

The MUC F-measure counts links shared between the system and gold partitions. Luo showed that this makes it structurally biased: it cannot distinguish system outputs of clearly different quality and can award higher scores to worse systems, because merging everything into one chain maximizes link recall at little precision cost ^[15]. B-cubed instead computes precision and recall per mention and takes a weighted sum over all mentions ^[1]. CEAF, the Constrained Entity-Alignment F-Measure, aligns each system entity with at most one gold entity before scoring ^[15].

The headline number reported in nearly all papers, "CoNLL F1", is the unweighted average of the MUC, B-cubed, and CEAF-phi4 F1 scores, an aggregate adopted by the CoNLL shared tasks ^[7]. Because small implementation differences change the numbers, the community standardized on a reference scorer released in 2014 rather than reimplementations ^[16].

Winograd schemas and commonsense

Terry Winograd's 1972 doctoral thesis contained the example "The city councilmen refused the demonstrators a permit because they feared/advocated violence", which showed that a single word change can flip the preferred antecedent in a way that requires world knowledge ^[1][19]. Hector Levesque, Ernest Davis, and Leora Morgenstern formalized this into the Winograd Schema Challenge, proposed as an alternative to the Turing test. A schema is a pair of sentences differing in one or two words, containing a referential ambiguity resolved in opposite directions in the two versions, with an answer that is obvious to a human reader but resistant to selectional restrictions or corpus statistics. The canonical example is "The trophy doesn't fit in the brown suitcase because it's too big/small. What is too big/small?" ^[17]. The original collection contained 273 expert-crafted problems ^[18].

The challenge did not survive the arrival of large pretrained models. By 2019, transformer models fine-tuned on Winograd-style data exceeded 90 percent accuracy, and a 2023 retrospective by the original organizers and collaborators concluded that the challenge had been defeated, while arguing that the surrogate-task methodology it pioneered remains instructive ^[19]. WinoGrande, released in 2019, was the direct response: 44,000 crowdsourced problems filtered adversarially to remove dataset artifacts, on which the best systems of the time scored 59.4 to 79.1 percent against human performance of 94.0 percent ^[18]. A natural language inference recasting of Winograd schemas, WNLI, was folded into the GLUE benchmark ^[1].

Gender and social bias

Because coreference systems must resolve gendered pronouns, they are a convenient probe for social bias. WinoBias, introduced in 2018, pairs Winograd-style sentences with occupations that carry gender stereotypes and measures whether a system does better on pro-stereotypical links than anti-stereotypical ones. Rule-based, feature-based, and neural systems all showed the pattern, with an average gap of 21.1 F1 between the two conditions ^[20].

GAP, released the same year, took a different approach: 8,908 ambiguous pronoun-name pairs drawn from 4,454 naturally occurring Wikipedia contexts, balanced by gender. Existing corpora were skewed, with fewer than 20 percent of the gendered pronouns in the English OntoNotes development and training sets being feminine. The best baseline at release reached 66.9 percent F1, and systems resolved feminine pronouns significantly worse than masculine ones ^[1][21]. Fine-tuning BERT-large into the coarse-to-fine architecture raised GAP overall F1 to 85.0 with a feminine-to-masculine ratio of 0.95 ^[4], and Maverick reports 91.1 ^[14]. Work in 2025 extended the paradigm to intersectional identity: WinoIdentity augments WinoBias with 25 demographic markers across 10 attributes for 245,700 prompts, and found confidence disparities as high as 40 percent across attributes including body type, sexual orientation, and socio-economic status, with the largest uncertainty on doubly disadvantaged identities in anti-stereotypical settings ^[22].

The large language model era

Whether coreference resolution still needs to be a separate component is an open question. Instruction-tuned models can be prompted to resolve coreference and outperform unsupervised systems, and they generalize surprisingly well across domains, languages, and time periods, but their accuracy depends heavily on the quality of mention detection, and fine-tuned task-specific models remain preferable when even a small annotated dataset is available ^[23]. A 2025 study found that models can perform coreference disambiguation well and can detect referential ambiguity well, but not both at once, a tension its authors call the correct-detect trade-off ^[24].

The multilingual shared tasks give the clearest running scoreboard. The fourth Shared Task on Multilingual Coreference Resolution, held at CODI-CRAC 2025, introduced a dedicated large language model track with a simplified plaintext input format and ran on CorefUD 1.3, a harmonized collection of 22 datasets in 17 languages. Nine systems took part, four of them LLM-based, and traditional systems kept the lead ^[25]. The fifth edition, at CODI-CRAC 2026, emphasized long-range entities, expanded to CorefUD 1.4 with 27 datasets in 19 languages, and drew ten systems including four LLM-based entries. Traditional systems again finished ahead, though the organizers noted that LLMs may challenge them in future editions ^[26]. CorefUD itself standardizes coreference annotation on top of the Universal Dependencies format so that corpora in different languages can be trained and evaluated together ^[27].

Document length is the other frontier. BOOKCOREF, published in 2025, is the first book-scale benchmark, with documents averaging more than 200,000 tokens. Training on it lets long-document systems gain up to 20 CoNLL F1 points when evaluated on full books, but models still fall well short of the accuracy they reach on short documents ^[28].

Applications and limitations

Coreference resolution is rarely an end in itself. It feeds information extraction, entity linking, relation extraction, knowledge graph construction, question answering, machine translation, and text summarization ^[14]. It has also found a use inside retrieval-augmented generation pipelines, where resolving pronouns in retrieved passages improves both retrieval relevance and answer quality, with smaller models benefiting most because they have less capacity to handle referential ambiguity on their own ^[29].

The main limitations are structural rather than incidental. OntoNotes does not annotate singletons, so a system tuned to it learns a version of the task that differs from what downstream applications need. The benchmark is more than a decade old and drawn from a fixed set of genres (newswire, broadcast news and conversation, magazine, web, and telephone conversation), and gains on it have been incremental for years ^[7]. Bias probes show that systems inherit stereotype associations from their training data and their embeddings ^[20][22]. And book-scale and low-resource multilingual settings remain far from solved ^[26][28].

See also

• Natural language processing
• Named entity recognition
• Winograd Schema Challenge
• WinoGrande
• BERT
• Commonsense reasoning

References

1. ^Daniel Jurafsky and James H. Martin, "Coreference Resolution and Entity Linking", chapter 26 of Speech and Language Processing, 3rd edition draft of September 23, 2023. web.stanford.edu/...26.pdf
2. ^Kenton Lee, Luheng He, Mike Lewis, Luke Zettlemoyer, "End-to-end Neural Coreference Resolution", EMNLP 2017, pages 188-197. aclanthology.org/D17-1018
3. ^Kenton Lee, Luheng He, Luke Zettlemoyer, "Higher-order Coreference Resolution with Coarse-to-fine Inference", NAACL 2018. arxiv.org/...1804.05392
4. ^Mandar Joshi, Omer Levy, Daniel S. Weld, Luke Zettlemoyer, "BERT for Coreference Resolution: Baselines and Analysis", EMNLP-IJCNLP 2019, pages 5803-5808. aclanthology.org/D19-1588
5. ^Ralph Grishman and Beth Sundheim, "Message Understanding Conference-6: A Brief History", COLING 1996. aclanthology.org/C96-1079
6. ^Linguistic Data Consortium, "OntoNotes Release 5.0" (LDC2013T19), released October 16, 2013. catalog.ldc.upenn.edu/LDC2013T19
7. ^Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, Yuchen Zhang, "CoNLL-2012 Shared Task: Modeling Multilingual Unrestricted Coreference in OntoNotes", EMNLP-CoNLL Shared Task 2012, pages 1-40. aclanthology.org/W12-4501
8. ^Sameer Pradhan, Lance Ramshaw, Mitchell Marcus, Martha Palmer, Ralph Weischedel, Nianwen Xue, "CoNLL-2011 Shared Task: Modeling Unrestricted Coreference in OntoNotes", CoNLL 2011, pages 1-27. aclanthology.org/W11-1901
9. ^Heeyoung Lee, Angel Chang, Yves Peirsman, Nathanael Chambers, Mihai Surdeanu, Dan Jurafsky, "Deterministic Coreference Resolution Based on Entity-Centric, Precision-Ranked Rules", Computational Linguistics 39(4), 2013, pages 885-916. aclanthology.org/J13-4004
10. ^Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, Omer Levy, "SpanBERT: Improving Pre-training by Representing and Predicting Spans", TACL, 2020. arxiv.org/...1907.10529
11. ^Vladimir Dobrovolskii, "Word-Level Coreference Resolution", EMNLP 2021. aclanthology.org/2021.emnlp-main.605
12. ^Shon Otmazgin, Arie Cattan, Yoav Goldberg, "LingMess: Linguistically Informed Multi Expert Scorers for Coreference Resolution", EACL 2023. arxiv.org/...2205.12644
13. ^Bernd Bohnet, Chris Alberti, Michael Collins, "Coreference Resolution through a seq2seq Transition-Based System", Transactions of the Association for Computational Linguistics 11 (2023), pages 212-226. aclanthology.org/2023.tacl-1.13
14. ^Giuliano Martinelli, Edoardo Barba, Roberto Navigli, "Maverick: Efficient and Accurate Coreference Resolution Defying Recent Trends", ACL 2024, pages 13380-13394. aclanthology.org/2024.acl-long.722
15. ^Xiaoqiang Luo, "On Coreference Resolution Performance Metrics", HLT-EMNLP 2005, pages 25-32. aclanthology.org/H05-1004
16. ^Sameer Pradhan, Xiaoqiang Luo, Marta Recasens, Eduard Hovy, Vincent Ng, Michael Strube, "Scoring Coreference Partitions of Predicted Mentions: A Reference Implementation", ACL 2014, pages 30-35. aclanthology.org/P14-2006
17. ^Hector J. Levesque, Ernest Davis, Leora Morgenstern, "The Winograd Schema Challenge", Thirteenth International Conference on the Principles of Knowledge Representation and Reasoning (KR 2012), page 552. cdn.aaai.org/...4492-21843-1-PB.pdf
18. ^Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, Yejin Choi, "WinoGrande: An Adversarial Winograd Schema Challenge at Scale", arXiv:1907.10641, 2019. arxiv.org/...1907.10641
19. ^Vid Kocijan, Ernest Davis, Thomas Lukasiewicz, Gary Marcus, Leora Morgenstern, "The Defeat of the Winograd Schema Challenge", arXiv:2201.02387. arxiv.org/...2201.02387
20. ^Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, Kai-Wei Chang, "Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods", NAACL 2018. arxiv.org/...1804.06876
21. ^Kellie Webster, Marta Recasens, Vera Axelrod, Jason Baldridge, "Mind the GAP: A Balanced Corpus of Gendered Ambiguous Pronouns", arXiv:1810.05201, 2018. arxiv.org/...1810.05201
22. ^Falaah Arif Khan, Nivedha Sivakumar, Yinong Oliver Wang, Katherine Metcalf, Cezanne Camacho, Barry-John Theobald, Luca Zappella, Nicholas Apostoloff, "Investigating Intersectional Bias in Large Language Models using Confidence Disparities in Coreference Resolution", arXiv:2508.07111, August 2025. arxiv.org/...2508.07111
23. ^Nghia T. Le and Alan Ritter, "Are Large Language Models Robust Coreference Resolvers?", arXiv:2305.14489, 2023. arxiv.org/...2305.14489
24. ^Amber Shore, Russell Scheinberg, Ameeta Agrawal, So Young Lee, "Correct-Detect: Balancing Performance and Ambiguity Through the Lens of Coreference Resolution in LLMs", arXiv:2509.14456, September 2025. arxiv.org/...2509.14456
25. ^Michal Novak, Miloslav Konopik, Anna Nedoluzhko, Martin Popel, Ondrej Prazak, Jakub Sido, Milan Straka, Zdenek Zabokrtsky, Daniel Zeman, "Findings of the Fourth Shared Task on Multilingual Coreference Resolution: Can LLMs Dethrone Traditional Approaches?", CODI-CRAC 2025. arxiv.org/...2509.17796
26. ^Michal Novak, Miloslav Konopik, Anna Nedoluzhko, Martin Popel, Ondrej Prazak, Jakub Sido, Milan Straka, Zdenek Zabokrtsky, Daniel Zeman, "Findings of the Fifth Shared Task on Multilingual Coreference Resolution: Expanding Datasets for Long-Range Entities", CODI-CRAC 2026. arxiv.org/...2605.21369
27. ^Anna Nedoluzhko, Michal Novak, Martin Popel, Zdenek Zabokrtsky, Amir Zeldes, Daniel Zeman, "CorefUD 1.0: Coreference Meets Universal Dependencies", LREC 2022, pages 4859-4872. aclanthology.org/2022.lrec-1.520
28. ^Giuliano Martinelli, Tommaso Bonomo, Pere-Lluis Huguet Cabot, Roberto Navigli, "BOOKCOREF: Coreference Resolution at Book Scale", arXiv:2507.12075, July 2025. arxiv.org/...2507.12075
29. ^Youngjoon Jang, Seongtae Hong, Junyoung Son, Sungjin Park, Chanjun Park, Heuiseok Lim, "From Ambiguity to Accuracy: The Transformative Effect of Coreference Resolution on Retrieval-Augmented Generation systems", arXiv:2507.07847, July 2025. arxiv.org/...2507.07847