AI Infrastructure

Explore AI Infrastructure through related topics and the articles other pages reference most.

Explore articles

Reset filters
Browse subtopics: Training & Optimization

Articles that also belong to these categories. Counts cover all of AI Infrastructure.

Showing 1-11 of 11 articles

Context Parallelism

Context Parallelism (CP) is a distributed training strategy that partitions the input sequence dimension of a transformer across multiple accelerators and uses ring-style point-to-point communication to…

Training & Optimization

DeepSpeed

DeepSpeed is an open-source deep learning optimization library, originally developed by Microsoft, that makes distributed training and inference of large models efficient, easy to use, and cost-effective.

Deep LearningMachine Learning

Olmo-core

Olmo-core is an open-source Python library for modeling and large-scale distributed training with PyTorch. The Allen Institute for AI (Ai2) develops and maintains it under the Apache 2.0 license.

Developer ToolsOpen Source AI

Tensor Core

A Tensor Core is a specialized execution unit inside NVIDIA GPUs that computes a small matrix multiplication and accumulation, D = A x B + C

AI ComputeAI Hardware