Saturday, July 11, 2026
- Layer normalizationv7Layer normalization is a technique for normalizing the activations of a neural network across the feature dimension of each individual sample, rather than...
- Step sizev5See also: learning rate, parameter update, Adam, SGD In machine learning, the step size (also called the learning rate, usually written as the Greek letter ...
- User matrixv6See also: Machine learning terms In collaborative filtering and matrix factorization recommender systems, the user matrix (commonly written U or P) is the...
- Model Parallelismv8See also: Machine learning terms, Data parallelism, GPU computing, Deep learning Model parallelism is a distributed training and inference technique that...
- Reinforcement Learning from Human Feedback (RLHF)v9Reinforcement Learning from Human Feedback (RLHF) is a machine learning technique that trains artificial intelligence systems to behave according to human...
- Hyperparameterv11See also: Machine learning terms A hyperparameter is a configuration setting in a machine learning algorithm that is fixed by the practitioner before training...
- Exploding Gradient Problemv11See also: Machine learning terms, Vanishing gradient problem, Gradient clipping, Backpropagation The exploding gradient problem is a training failure in deep...
- Criticv7A critic in reinforcement learning (RL) is the component of an actor-critic system that estimates a value function, scoring how good the actor's chosen actions...
- L2 Lossv4L2 loss is the squared-error loss function: for a true value and a predicted value , it is the squared difference , and averaging it across a dataset gives...
- State (Reinforcement Learning)v7In reinforcement learning (RL), a state is a complete description of the environment at a particular point in time, containing all the information an agent...
- DeepNorm / DeepNetv2DeepNorm is a normalization and weight initialization scheme for Transformer networks that makes the training of very deep models stable. It was introduced by...
- EnCodecv4EnCodec is a real-time neural audio codec developed by Meta AI's FAIR (Fundamental AI Research) team that compresses speech, ambient sound, and music into a...
- Sparse Codingv3Sparse coding is a representation learning principle in which a signal is encoded as a linear combination of a small number of elements drawn from a (usually...
- AdaBoostv3AdaBoost (short for Adaptive Boosting) is a machine learning ensemble algorithm that combines many weak classifiers into a single strong classifier through a...
- k-Nearest Neighborsv6k-Nearest Neighbors (often abbreviated k-NN or KNN) is a non-parametric, instance-based supervised learning algorithm that classifies a new data point by...
- Recursive reward modelingv2Recursive reward modeling (RRM) is a proposed approach to the scalable oversight problem in AI alignment, in which agents trained by reward modeling are...
- Florence-2v3Florence-2 is a vision foundation model developed by Microsoft Research that handles a wide range of computer vision and vision-language tasks through a single...
- Evan Hubingerv3Evan Hubinger is an American AI safety researcher who leads the alignment stress-testing team at Anthropic, where he serves as a Member of Technical Staff and...
- BitNet b1.58v3BitNet b1.58 is a ternary-weight large language model architecture from Microsoft Research in which every weight is constrained to one of three values, -1, 0,...
- Bias-variance tradeoffv8The bias-variance tradeoff is a foundational concept in machine learning and statistics that describes the tension between two competing sources of error in...
- Decoding strategiesv5Decoding strategies are the algorithms that select output tokens from a language model's next-token probability distribution during text generation. At each...
- Expectation-Maximization (EM) Algorithmv7The Expectation-Maximization (EM) algorithm is an iterative method for finding maximum likelihood or maximum a posteriori (MAP) estimates of the parameters of...
- Grad-CAMv4Grad-CAM (Gradient-weighted Class Activation Mapping) is a technique for producing visual explanations from convolutional neural network (CNN) models by using...
- Grouped-Query Attentionv8Grouped-query attention (GQA) is an attention mechanism for transformer language models that partitions the query heads into a small number of groups, where...
- Rotary Position Embeddingv10Rotary Position Embedding (RoPE) is a positional encoding method for transformer models that encodes a token's absolute position by rotating its query and key...
- Sigmoid Functionv11See also: Machine learning terms The sigmoid function is a mathematical function that maps any real number to a value between 0 and 1 using the formula ,...
- Online learningv6See also: Machine learning terms Online learning is a machine learning paradigm in which a model receives data sequentially, one example or one mini-batch at a...
- Loss Functionv11See also: Machine learning terms A loss function is a mathematical formula that measures how far a machine learning model's prediction is from the correct...
- Cross-Entropyv6See also: Machine learning terms, Loss function, Entropy Cross-entropy is a measure from information theory of how many bits (or nats) are needed to encode...
- Entropyv5Entropy is a quantitative measure of the uncertainty, randomness, or average information content in a probability distribution: for a discrete random variable...
- Pruningv7Pruning is a family of techniques used in machine learning and artificial intelligence to remove parts of a model or search space that are estimated to be...
- ZeRO (Zero Redundancy Optimizer)v3ZeRO (Zero Redundancy Optimizer) is a family of memory-optimization techniques for training large neural networks introduced by Samyam Rajbhandari, Jeff...
- Cosine similarityv5Cosine similarity is a measure of similarity between two non-zero vectors that calculates the cosine of the angle between them, defined as the dot product of...
- ROC (Receiver Operating Characteristic) Curvev7A Receiver Operating Characteristic (ROC) curve is a graph that measures how well a binary classification system separates two classes by plotting its true...
- Multinomial regressionv4See also: Machine learning terms Multinomial regression is a statistical model that predicts which one of K possible categories an observation belongs to,...
- Logitsv7In machine learning and statistics, logits refer to the raw, unnormalized scores output by a model before they are converted into probabilities. The term has...
- Least Squares Regressionv5Least squares regression is a statistical method that fits a model to data by choosing the parameters that minimize the sum of the squared residuals, the...
- Classification Thresholdv6A classification threshold (also called a decision threshold or cut-off point) is a numeric value used to convert the continuous probability output of a...
- DDIM (Denoising Diffusion Implicit Models)v4Denoising Diffusion Implicit Models (DDIM) are a class of iterative generative models, introduced by Jiaming Song, Chenlin Meng, and Stefano Ermon of Stanford...
- Adafactorv3Adafactor is a memory-efficient adaptive learning-rate optimizer for training deep neural networks, introduced by Noam Shazeer and Mitchell Stern in the 2018...
- Principal Component Analysis (PCA)v6Principal component analysis (PCA) is an unsupervised learning technique for dimensionality reduction that identifies the orthogonal directions of maximum...
- Adversarial attackv7An adversarial attack is a technique for crafting inputs that are deliberately designed to cause artificial intelligence systems, particularly machine learning...
- Kolmogorov-Arnold Networkv4A Kolmogorov-Arnold Network (KAN) is a type of neural network architecture proposed as an alternative to the traditional Multi-Layer Perceptron (MLP)....
- Target Networkv5A target network is a separate, slowly updated copy of a neural network used in deep reinforcement learning to compute stable learning targets, decoupling the...
- Elastic Netv4See also: Regularization, Linear regression Elastic Net is a regularization and variable selection method for linear regression and other generalized linear...
- Temperature samplingv5Temperature sampling is the most common decoding control in large language models: a single hyperparameter, written T, that divides the model's output logits...
- SHAP (SHapley Additive exPlanations)v5See also: explainable AI, feature importance, LIME, permutation feature importance SHAP (SHapley Additive exPlanations) is a game-theoretic method that...
- Adam optimizerv6The Adam optimizer (short for Adaptive Moment Estimation) is an algorithm for first-order gradient descent-based optimization of stochastic objective...
- Sparse attentionv6Sparse attention is a family of techniques that cut the computational and memory cost of the attention mechanism in transformer models by letting each token...
- Full Softmaxv6Full softmax (also called the standard softmax or exact softmax) is the softmax computation that calculates a probability for every possible output class in a...
- AUC-ROCv9See also: Machine learning terms AUC (Area Under the Curve), most often the area under the ROC curve (AUC-ROC), is a threshold-independent evaluation metric...
- SARSA (State-Action-Reward-State-Action)v3SARSA (State-Action-Reward-State-Action) is an on-policy temporal-difference (TD) control algorithm that learns the action-value function Q^pi(s, a) of the...
- Earth Mover's Distancev6Earth Mover's Distance (EMD), also known as the Wasserstein-1 distance, Kantorovich-Rubinstein metric, or Mallows's distance, is a measure of dissimilarity...
- Recurrent Neural Networkv10See also: Machine learning terms A recurrent neural network (RNN) is a class of artificial neural network designed to process sequential data by maintaining an...
- Statistical learning theoryv4Statistical learning theory (SLT) is the mathematical framework that explains when and why machine learning algorithms generalize from a finite training sample...
- Voltage Parkv3Voltage Park is a United States cloud computing company that operates a fleet of NVIDIA H100 graphics processing units for artificial intelligence training and...
- Open-Sorav3Open-Sora is an open-source text-to-video diffusion project initiated in March 2024 by Singapore-based startup HPC-AI Tech (the team behind the Colossal-AI...
- Loss Surfacev4The loss surface (also called the loss landscape, the error surface, or the objective function surface) is the geometric representation of a loss function as a...
- LLaMA-Factoryv3LLaMA-Factory is an open-source unified framework for the efficient fine-tuning of large language models (LLMs) and vision-language models (VLMs). It...
- GitHub Copilot Xv6GitHub Copilot X was the umbrella name GitHub used in 2023 for its vision of an artificial intelligence assistant that would reach across the entire developer...
- MiniCPM-Vv3MiniCPM-V is a family of open-weights multimodal large language models developed by the OpenBMB lab at Tsinghua University's Natural Language Processing group...
- GPU clusterv4A GPU cluster is a tightly coupled collection of servers, each populated with multiple graphics processing units, interconnected by high-bandwidth low-latency...
- Deepgram Nova-3v3Deepgram Nova-3 is the third-generation automatic speech recognition (ASR) model developed by Deepgram, a San Francisco-based voice AI company. The model...
- Blueprint for an AI Bill of Rightsv3The Blueprint for an AI Bill of Rights is a non-binding policy framework released by the White House Office of Science and Technology Policy (OSTP) on October...
- Agentic misalignmentv2Agentic misalignment is a term coined by Anthropic in a June 2025 research release for cases in which a goal-directed large language model (LLM), placed in an...
- Fully Connected Layerv5A fully connected layer (also called a dense layer or linear layer) is a layer in an artificial neural network in which every input value connects to every...
- Gaussian Processv3A Gaussian process (GP) is a probabilistic machine learning model defined as a collection of random variables, any finite number of which have a joint Gaussian...
- Importance samplingv4Importance sampling (often abbreviated IS) is a Monte Carlo method for estimating the expectation of a function under a target probability distribution by...
- Weight Decayv4Weight decay is a regularization technique used in training neural networks that shrinks every parameter toward zero by a small fraction on each update step,...
- DeepSpeedv9DeepSpeed is an open-source deep learning optimization library developed by Microsoft that makes distributed training and inference of large models efficient,...
- One-vs.-allv5See also: Machine learning terms One-vs.-all (OvA), also known as one-vs.-rest (OvR) or one-against-all, is a strategy for turning a multi-class classification...
- Decision Forestv5See also: Random Forest, Decision Tree, Ensemble Learning A decision forest is a family of ensemble learning methods in machine learning that combine many...
- Attribute samplingv4See also: Machine learning terms Attribute sampling is a randomization technique in which a decision tree considers only a small, randomly drawn subset of the...
- Activation Functionv7An activation function is a nonlinear mathematical function applied to the output of each neuron in an artificial neural network, and it is what gives the...
- Pudu FlashBot Armv5--- General information Pudu Robotics Country of origin 2025 Status Available (Q1 2026 onward) Website The Pudu FlashBot Arm (stylized as PUDU...
- MetaGPTv3MetaGPT is an open-source multi-agent framework that organizes large language model agents into a simulated software-development company, with role-specialized...
- LimX Oliv8--- General information LimX Dynamics Country of origin 2025 Status RMB 158,000 ($21,800 USD) Height 55 kg (121 lbs) Degrees of freedom 5 km/h...
- HuggingFace TRLv3TRL (Transformer Reinforcement Learning, now stylized as Transformers Reinforcement Learning) is an open-source Python library maintained by Hugging Face for...
- EXAONEv2EXAONE (an acronym for EXpert AI for EveryONE) is the family of large language models and foundation models developed by LG AI Research, the artificial...
- Attention Is All You Needv9"Attention Is All You Need" is the 2017 research paper that introduced the transformer, the neural network architecture that underpins virtually every modern...
- Vanishing Gradient Problemv8See also: Machine learning terms The vanishing gradient problem is a difficulty in training deep neural networks where the gradients used to update the network...
- Momentumv6See also: Machine learning terms, Optimizer, Gradient descent Momentum is an optimization technique that accelerates gradient descent by accumulating an...
- Convergencev5Convergence in machine learning is the point at which an iterative optimization algorithm reaches a stable solution, meaning the loss function stops decreasing...
- Episode (Reinforcement Learning)v6An episode in reinforcement learning is one complete sequence of interaction between an agent and its environment, starting from an initial state and ending...
- YaRNv5YaRN (Yet another RoPE extensioN) is a compute-efficient method for extending the context window of large language models that use Rotary Position Embeddings...
- SwiGLUv5SwiGLU (Swish-Gated Linear Unit) is the activation function used inside the feed-forward sublayer of most modern transformer large language models, including...
- Markov chainv4A Markov chain is a stochastic process in which the probability of the next state depends only on the current state and not on the sequence of states that came...
- Temporal-difference learningv4Temporal-difference (TD) learning is a class of model-free reinforcement learning methods that learn value-function estimates by bootstrapping: updating each...
- DDPMv9Denoising Diffusion Probabilistic Models (DDPM) are a class of generative model introduced by Jonathan Ho, Ajay Jain, and Pieter Abbeel of UC Berkeley in their...
- Q-Functionv7The Q-function, also called the action-value function or state-action value function and written , is the function in reinforcement learning (RL) that returns...
- Random Forestv10See also: Machine learning terms A random forest is a supervised machine learning algorithm that builds many decision trees on random subsets of the data and...
- Dense Layerv7A dense layer, also called a fully connected (FC) layer, linear layer, or affine layer, is a layer in an artificial neural network where every input neuron is...
- Attentionv15See also: Machine learning terms > This article gives a high-level overview of attention as a family of mechanisms in machine learning. For deeper treatments,...
- Singular value decompositionv7See also: principal component analysis, eigenvalue decomposition, linear algebra, matrix factorization, pseudoinverse, low-rank adaptation, Eckart-Young...
- Discount Factorv4The discount factor, almost always written as the Greek letter (gamma), is a scalar hyperparameter in reinforcement learning that controls how much an agent...
- Trajectory (Reinforcement Learning)v6A trajectory in reinforcement learning is a sequence of states, actions, and rewards that an agent experiences while interacting with an environment. Formally...
- Markov propertyv9The Markov property is the condition that a stochastic process is memoryless: the future of the process depends only on its present state, not on the path the...
- Prologv2Prolog (a contraction of the French PROgrammation en LOGique) is a logic programming language, first implemented in 1972 by Alain Colmerauer's group at the...
- NuminaMathv4NuminaMath is a family of openly licensed competition-mathematics resources developed by the non-profit Project Numina, spanning the largest public dataset of...
- Fairlearnv2Fairlearn is an open-source Python toolkit for assessing and improving the fairness of machine-learning models with respect to sensitive attributes such as...
