Panoptic Segmentation

Panoptic segmentation is a computer vision task that assigns every pixel in an image a semantic class label and, for countable objects, an instance identity. It unifies the two segmentation problems that the field had studied separately for years: semantic segmentation, which labels amorphous "stuff" regions such as sky, road, and grass without distinguishing individual instances, and instance segmentation, which detects and delineates each countable "thing" such as a person or a car but ignores background regions. A panoptic model must produce a single, coherent, non-overlapping scene decomposition covering both ^[1].

The task was formalized by Alexander Kirillov, Kaiming He, Ross Girshick, Carsten Rother, and Piotr Dollar in a paper first posted in January 2018 and published at CVPR 2019 ^[1]. Along with the task definition, the paper introduced the panoptic quality (PQ) metric, which scores stuff and thing classes in a uniform way and decomposes into a segmentation term and a recognition term. COCO and Mapillary Vistas ran panoptic challenge tracks at ECCV 2018 ^[1], panoptic evaluation quickly became standard on Cityscapes and ADE20K as well, and by 2021-2022 the task had become the proving ground for universal segmentation architectures such as MaskFormer, Mask2Former, and kMaX-DeepLab, which handle semantic, instance, and panoptic segmentation with a single design ^[7][8][9].

Background: two segmentation literatures

Before 2018, image segmentation research ran on parallel tracks. Semantic segmentation grew out of scene parsing: fully convolutional networks assign one class label per pixel, treating a crowd of people as an undifferentiated "person" region. Instance segmentation grew out of object detection: detectors such as Mask R-CNN output a mask per detected object, with overlaps allowed and background classes ignored. Each track had its own datasets, metrics (mIoU for semantic, AP for instance), and architectural conventions, and methods from one track were rarely evaluated on the other ^[1].

Kirillov and colleagues argued that this schism left neither task matching what applications such as autonomous driving or augmented reality actually need: a complete, consistent labeling of the whole scene. Their paper set out to "revive the interest of the community in a more unified view of image segmentation" by defining the joint task and, critically, a metric that could evaluate it ^[1].

Task definition

In the panoptic format, each pixel of an image is mapped to a pair: a semantic label and an instance id ^[1]. For stuff classes the instance id is ignored, so all sky pixels form one segment. For thing classes, pixels sharing both label and id form one object instance. Two properties distinguish the output from instance segmentation:

• Segments may not overlap. Each pixel belongs to exactly one segment, so the prediction is a partition of the image rather than a ranked list of possibly overlapping masks.
• There is no confidence ranking. The model commits to one scene interpretation, which mirrors how the ground truth itself is annotated.

The COCO panoptic tooling stores all segments of an image in a single PNG, a format that enforces the non-overlap property by construction ^[16].

The panoptic quality metric

PQ is computed per class and then averaged. A predicted segment matches a ground-truth segment only if their intersection over union (IoU) is strictly greater than 0.5; the paper proves this threshold makes the matching unique, so no assignment optimization is needed ^[1]. Given the resulting sets of true positives (TP), false positives (FP), and false negatives (FN):

PQ = (sum of IoU over matched pairs) / (|TP| + 0.5|FP| + 0.5|FN|)

The score factors into two interpretable terms, PQ = SQ x RQ, where segmentation quality (SQ) is the average IoU of matched segments and recognition quality (RQ) is an F1-style detection score, |TP| / (|TP| + 0.5|FP| + 0.5|FN|) ^[1].

To calibrate the metric, the authors measured human consistency on images annotated twice, obtaining PQ between 57.5 and 69.7 depending on the dataset. Human consistency was close between stuff and things on Cityscapes and ADE20K (with a somewhat larger gap on Vistas), while machine results of the time trailed humans mainly on RQ, indicating that recognition rather than boundary quality was the dominant error source ^[1].

Datasets and benchmarks

The original study used three datasets that already carried both semantic and instance annotations; COCO added a dedicated panoptic challenge in 2018, with annotations covering the 80 thing categories of the detection task plus 53 stuff categories ^{[1][8][16][19]}.

Scroll sideways for more →

Methods

Two-branch pipelines

The first wave of panoptic systems bolted the two older task formulations together. Panoptic FPN (Kirillov et al., CVPR 2019) added a lightweight semantic segmentation branch to Mask R-CNN on a shared feature pyramid network backbone, giving a single network that emits both outputs; the authors positioned it as a simple, strong baseline rather than a new architecture ^[2]. UPSNet (Xiong et al., CVPR 2019) went a step further with a parameter-free panoptic head that fuses the semantic and instance logits and predicts an extra "unknown" class to resolve conflicts between the two branches ^[3].

These proposal-based ("top-down") designs required heuristic merging: overlapping instance masks had to be flattened and reconciled with the stuff map. Panoptic-DeepLab (Cheng et al., CVPR 2020) showed a bottom-up alternative could compete: a convolutional neural network with dual decoders predicts a semantic map plus class-agnostic instance centers and offsets, and pixels are grouped to their nearest center. At publication it ranked first on all three Cityscapes benchmarks (84.2 mIoU, 39.0 AP, 65.5 PQ on the test set), and a MobileNetV3 variant ran near real time at 15.8 frames per second ^[4].

Unified mask transformers

DETR (Carion et al., 2020) reframed detection as set prediction with a transformer and showed the same model "can be easily generalized to produce panoptic segmentation in a unified manner" by adding a mask head ^[5]. MaX-DeepLab (Wang et al., CVPR 2021) then removed the remaining hand-designed components and became, in its authors' words, "the first end-to-end model for panoptic segmentation," predicting class-labeled masks directly and reaching 51.3 PQ on COCO test-dev without test-time augmentation ^[6].

MaskFormer (Cheng, Schwing, and Kirillov, NeurIPS 2021) distilled the lesson into a general recipe: treat every segmentation task as mask classification, predicting a set of binary masks each paired with one class label. The same model handles semantic and panoptic segmentation, scoring 55.6 mIoU on ADE20K and 52.7 PQ on COCO ^[7]. Its successor Mask2Former (CVPR 2022) added masked attention, which restricts cross-attention to the region of each predicted mask, and set new highs across tasks: 57.8 PQ on COCO panoptic, 50.1 AP on COCO instance, and 57.7 mIoU on ADE20K semantic segmentation ^[8]. Both models were developed at Facebook AI Research (now Meta AI) with academic collaborators and are widely used through Detectron2 and the Hugging Face Transformers library.

kMaX-DeepLab (Yu et al., ECCV 2022) reinterpreted the cross-attention between object queries and pixels as a k-means clustering step, arguing that attention designs inherited from language modeling fit poorly with the very long pixel sequences of vision transformer style models. It reached 58.0 PQ on COCO val, 68.4 PQ on Cityscapes val, and 50.9 PQ on ADE20K val ^[9]. OneFormer (Jain et al., CVPR 2023) closed the loop on universality: a task-conditioned joint training scheme with a task token lets one set of weights serve semantic, instance, and panoptic inference, outperforming per-task Mask2Former models on ADE20K, Cityscapes, and COCO while training only once ^[10][11].

Scroll sideways for more →

PQ figures use different backbones (for example Swin Transformer, ConvNeXt, or DiNAT variants) and evaluation splits, so the table indicates the trajectory rather than a strict like-for-like ranking.

Relation to SAM and open-vocabulary models

The Segment Anything Model (SAM, 2023) is sometimes described as making dedicated segmentation models obsolete, but it addresses a different problem. SAM performs promptable segmentation: given a point, box, or mask prompt it returns class-agnostic masks, trained on the SA-1B dataset of over one billion masks on 11 million images ^[12]. It does not assign category labels, and its authors list this as an open limitation: "it is unclear how to design simple prompts that implement semantic and panoptic segmentation" ^[12]. A SAM output is therefore not a panoptic result; it lacks the semantic labels and the guaranteed complete, non-overlapping scene partition that PQ evaluates. SAM 2 extends prompting to video but keeps the class-agnostic formulation.

A separate line of work removes the fixed class vocabulary instead. Open-vocabulary panoptic segmentation trains on one labeled vocabulary and evaluates on unseen category sets, typically by pairing a mask generator with CLIP-style image-text embeddings. FC-CLIP (NeurIPS 2023) builds both mask generation and classification on a single frozen convolutional CLIP backbone and, trained only on COCO panoptic data, reaches 26.8 PQ on ADE20K and 44.0 PQ on Cityscapes in zero-shot transfer ^[13]. OMG-Seg (CVPR 2024) pushes consolidation further, handling more than ten tasks in one model, including image and video panoptic segmentation, open-vocabulary settings, and SAM-like interactive prompting ^[14]. The absolute open-vocabulary numbers remain far below vocabulary-specific training, which is why closed-vocabulary models such as Mask2Former and its descendants are still standard baselines.

Extensions and applications

Video panoptic segmentation (Kim et al., CVPR 2020) extends the task through time, requiring consistent segmentation plus association of instance ids across frames, evaluated by a video panoptic quality (VPQ) metric and supported by the Cityscapes-VPS and reformatted VIPER datasets ^[15]. The panoptic formulation also underpins scene understanding in driving stacks, where street-scene benchmarks (Cityscapes, Mapillary Vistas) were panoptic early adopters and where a complete labeling of drivable surface, lanes, and every road user in one output is more directly useful than separate semantic and instance maps ^[1][3][4]. Similar full-scene requirements appear in robotics, mapping, and aerial imagery.

For practitioners, pretrained panoptic models ship in mainstream libraries: Detectron2 provides Panoptic FPN baselines, the official Mask2Former implementation is built on Detectron2, and the OneFormer repository distributes pretrained checkpoints for all three tasks ^[8][11].

Limitations

PQ itself has known rough edges: it treats every class equally regardless of area, so a rare stuff class with a few mislabeled segments can move the average as much as systematic errors on large regions, and the hard 0.5 IoU matching threshold gives no credit for near misses. The original paper's human-consistency study also puts a practical ceiling on the task: even trained annotators agree with each other at only 57.5 to 69.7 PQ depending on the dataset, so headline numbers should be read against that reference, not against 100 ^[1]. Annotation cost is another constraint; producing complete, non-overlapping labelings of every pixel is more expensive than either constituent task, which is part of why panoptic ground truth exists for far fewer domains than semantic labels do.

See also

• Semantic segmentation
• Instance segmentation
• Image segmentation
• Object detection
• Segment Anything Model and dataset
• DETR

References

1. ^Kirillov, A., He, K., Girshick, R., Rother, C., Dollar, P. "Panoptic Segmentation." CVPR 2019. arXiv:1801.00868. arxiv.org/...1801.00868
2. ^Kirillov, A., Girshick, R., He, K., Dollar, P. "Panoptic Feature Pyramid Networks." CVPR 2019. arXiv:1901.02446. arxiv.org/...1901.02446
3. ^Xiong, Y., Liao, R., Zhao, H., Hu, R., Bai, M., Yumer, E., Urtasun, R. "UPSNet: A Unified Panoptic Segmentation Network." CVPR 2019. arXiv:1901.03784. arxiv.org/...1901.03784
4. ^Cheng, B., Collins, M. D., Zhu, Y., Liu, T., Huang, T. S., Adam, H., Chen, L.-C. "Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation." CVPR 2020. arXiv:1911.10194. arxiv.org/...1911.10194
5. ^Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S. "End-to-End Object Detection with Transformers." 2020. arXiv:2005.12872. arxiv.org/...2005.12872
6. ^Wang, H., Zhu, Y., Adam, H., Yuille, A., Chen, L.-C. "MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers." CVPR 2021. arXiv:2012.00759. arxiv.org/...2012.00759
7. ^Cheng, B., Schwing, A. G., Kirillov, A. "Per-Pixel Classification is Not All You Need for Semantic Segmentation." NeurIPS 2021. arXiv:2107.06278. arxiv.org/...2107.06278
8. ^Cheng, B., Misra, I., Schwing, A. G., Kirillov, A., Girdhar, R. "Masked-attention Mask Transformer for Universal Image Segmentation." CVPR 2022. arXiv:2112.01527. arxiv.org/...2112.01527
9. ^Yu, Q., Wang, H., Qiao, S., Collins, M., Zhu, Y., Adam, H., Yuille, A., Chen, L.-C. "k-means Mask Transformer." ECCV 2022. arXiv:2207.04044. arxiv.org/...2207.04044
10. ^Jain, J., Li, J., Chiu, M., Hassani, A., Orlov, N., Shi, H. "OneFormer: One Transformer to Rule Universal Image Segmentation." CVPR 2023. arXiv:2211.06220. arxiv.org/...2211.06220
11. ^SHI-Labs. "OneFormer" (official repository, CVPR 2023 results). GitHub. github.com/...OneFormer
12. ^Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., Dollar, P., Girshick, R. "Segment Anything." 2023. arXiv:2304.02643. arxiv.org/...2304.02643
13. ^Yu, Q., He, J., Deng, X., Shen, X., Chen, L.-C. "Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIP." NeurIPS 2023. arXiv:2308.02487. arxiv.org/...2308.02487
14. ^Li, X., Yuan, H., Li, W., Ding, H., Wu, S., Zhang, W., Li, Y., Chen, K., Loy, C. C. "OMG-Seg: Is One Model Good Enough For All Segmentation?" CVPR 2024. arXiv:2401.10229. arxiv.org/...2401.10229
15. ^Kim, D., Woo, S., Lee, J.-Y., Kweon, I. S. "Video Panoptic Segmentation." CVPR 2020. arXiv:2006.11339. arxiv.org/...2006.11339
16. ^COCO Consortium. "COCO 2018 Panoptic Segmentation Task API." GitHub. github.com/...panopticapi
17. ^Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., Schiele, B. "The Cityscapes Dataset for Semantic Urban Scene Understanding." CVPR 2016. arXiv:1604.01685. arxiv.org/...1604.01685
18. ^Zhou, B., Zhao, H., Puig, X., Xiao, T., Fidler, S., Barriuso, A., Torralba, A. "Semantic Understanding of Scenes through the ADE20K Dataset." arXiv:1608.05442. arxiv.org/...1608.05442
19. ^COCO Consortium. "COCO Panoptic Segmentation Task." cocodataset.org