Condition

See also: Machine learning terms

In machine learning, a condition is any node in a decision tree that performs a test on one or more features and routes an example to one of its child nodes based on the result ^[1]. Google's Machine Learning Glossary defines a condition as "any node that performs a test" and notes that the terms condition, split, and test are synonyms in decision tree terminology ^[1]. Conditions are the working parts of a trained tree: every non-leaf node holds one, while the leaf nodes at the bottom hold only predictions.

Conditions are the building blocks of decision forests, the model family that includes random forests and gradient boosted trees ^[1]. During inference, an example follows what Google calls an inference path: it starts at the root condition, answers each test in turn, and ends at a leaf whose stored value is returned as the prediction ^[1][4].

The word "condition" also appears loosely elsewhere in machine learning, for example to describe rules in expert systems or stopping criteria in optimization. The strict definition used in this article is the modern decision tree one: a node that tests features and routes examples ^[1].

Where do conditions live in a decision tree?

A decision tree has three kinds of nodes ^[3]:

Scroll sideways for more →

In a decision tree diagram the root sits at the top and the leaves at the bottom ^[3]. A condition lives at the root and every internal node, so a binary tree with n leaves contains exactly n - 1 conditions. A condition does not store a prediction; it only decides which child an example goes to next ^[1].

Anatomy of a condition

Most conditions in practical tree libraries follow one of a few standard shapes. Google's Decision Forests course identifies five common forms ^[2]:

Scroll sideways for more →

Google's course states that the threshold condition is "the most frequently used type of condition" ^[2]. It is also what scikit-learn's DecisionTreeClassifier produces by default and what Google's YDF library uses out of the box ^[2][10]. The other forms matter when the data has many categories, when classes are not separable along single feature axes, or when missing values need their own path.

What is the difference between axis-aligned and oblique conditions?

Google's glossary draws a line between two families of conditions based on how many features they touch ^[1].

An axis-aligned condition involves exactly one feature. As Google puts it, "An axis-aligned condition involves only a single feature" ^[2]. A test like num_legs >= 2 is axis-aligned because it only looks at one column. Plotted in feature space, these conditions become splits parallel to one of the coordinate axes, which is where the name comes from. Most production libraries train axis-aligned trees by default because they are fast to learn, fast to evaluate, and easy to interpret ^[2].

An oblique condition "involves multiple features" ^[2]. The classic example is something like height > width, but in practice oblique splits are usually learned as linear combinations of several features at once, for example 0.4 * x1 + 0.6 * x2 >= threshold. The original algorithm to learn such splits is OC1, introduced by Murthy, Kasif, and Salzberg in their 1994 paper "A System for Induction of Oblique Decision Trees," published in the Journal of Artificial Intelligence Research, volume 2, pages 1-32 ^[5]. OC1 "combines deterministic hill-climbing with two forms of randomization to find a good oblique split (in the form of a hyperplane) at each node of a decision tree" ^[5], and it remains the canonical reference for multivariate decision trees.

Oblique trees can carve more flexible decision boundaries because their splits do not have to align with the feature axes. The trade-off is cost. Finding a good oblique condition is harder than scanning thresholds on a single feature, so training and inference are both slower ^[2]. In Google's YDF library this mode is opt-in, enabled with split_axis="SPARSE_OBLIQUE", which draws random sparse oblique splits over a small number of features following Tomita et al. (2020), "Sparse Projection Oblique Random Forests" ^[10][11].

What is the difference between binary and non-binary conditions?

A second axis of classification is the number of outgoing branches.

A binary condition has exactly two possible outcomes, typically yes and no. Decision trees that only contain binary conditions are called binary decision trees. Most modern libraries train binary trees because they are simpler to implement, less prone to overfitting, and easy to combine with techniques like surrogate splits for missing values.

A non-binary condition has more than two outcomes. A node that tests a categorical feature with three possible values could in principle branch three ways. CHAID, for example, learns multi-way splits for classification. Non-binary conditions are more expressive but also more prone to overfit, since each extra branch splits the training data into a smaller subset.

The two families are not as different as they look. As Google's Decision Forests course states, "A non-binary condition can be emulated with multiple binary conditions; therefore, binary trees are not inherently less powerful than non-binary trees" ^[2]. A binary tree can represent the same partitioning as a multi-way tree, just with more layers. That is one reason binary splits dominate in modern libraries.

How are conditions learned during training?

Conditions are not written by hand. They are picked by a part of the training algorithm called the splitter, which Google's glossary defines as "the routine (and algorithm) responsible for finding the best condition at each node" ^[1]. During training, the algorithm starts with all examples at the root and asks the splitter to find the best possible test for that node. Once the best condition is chosen, the examples are partitioned according to the condition's outcome, and the process repeats on each child ^[6].

This recursive procedure, called top-down induction of decision trees, is greedy. It picks the locally best condition at every node without backtracking, which means the final tree is not guaranteed to be globally optimal ^[6]. Finding the globally optimal tree is computationally intractable: Hyafil and Rivest proved in 1976 that constructing an optimal binary decision tree is NP-complete ^[12], so greedy growth is the standard compromise.

The quality of a candidate condition is measured by a splitting criterion. The choice of criterion depends on the task and the algorithm:

Scroll sideways for more →

The ID3 algorithm, introduced by J. Ross Quinlan in 1986, was a pioneering method that uses information gain based on entropy to select the best splitting attribute at each node ^[7]. CART, introduced by Leo Breiman and colleagues in their 1984 book Classification and Regression Trees, instead defaults to Gini impurity for classification and least-squares (variance reduction) for regression ^[8]. For a numeric feature with many possible thresholds, the splitter scans the candidate cutpoints (often the unique sorted values), scores each one by the chosen criterion, and keeps the best. For a categorical feature with k levels, exhaustive search over all partitions is exponential, so libraries use heuristics such as the Breiman two-class shortcut or random subset sampling. Google's YDF picks the splitter automatically based on feature type and hyperparameters, since the splitter is, in their words, the "bottleneck when training a decision tree" ^[6].

Worked examples

A spam classifier might use a root condition like num_links >= 10. This is an axis-aligned, binary, threshold condition on a single numeric feature. Emails that satisfy the test go down the yes branch toward more conditions about sender reputation or capitalized words; those that fail follow a different sub-tree.

A regression tree that predicts house prices might use square_footage > 2000 as its root. The two children are themselves conditions, perhaps num_bathrooms >= 3 and zip_code in {94110, 94114, 94117}. The latter is an in-set condition on a categorical feature. The leaves hold predicted prices in dollars rather than class labels.

On the classic Iris dataset, an axis-aligned tree typically splits on petal length and petal width. An oblique tree might learn a condition like 0.6 * petal_length + 0.4 * petal_width >= 3.1. The boundary is a diagonal line in feature space, which can separate the three species with fewer total splits ^[5].

How are conditions used inside ensembles?

In modern practice, single decision trees are rarely used on their own; they are almost always combined into ensembles. Conditions still play the same role inside each tree, but the way they are learned changes.

Random forests train many trees in parallel on different bootstrap samples of the data. At each node, the splitter is restricted to consider only a random subset of features. Google's glossary calls this attribute sampling: "a tactic for training a decision forest in which each decision tree considers only a random subset of possible features" ^[1]. The point is to make the trees less correlated with each other, which lowers the variance of the averaged prediction. Each tree's conditions are still axis-aligned thresholds in the default case, but the candidate features at any node are sampled rather than exhaustive.

Gradient boosted decision trees (GBDTs), used by libraries like XGBoost, LightGBM, and CatBoost, train trees sequentially. Each new tree fits the residual errors of the current ensemble. The conditions inside each tree are learned by a splitter, but the splitting criterion is based on the gradient and (often) the Hessian of the loss with respect to the current predictions, not on Gini or entropy directly. XGBoost, for example, uses a regularized objective and picks the condition that maximizes the gain in that objective ^[9]. The trees in a GBDT are usually shallow (a few levels) so that no single tree overfits.

Both approaches still rely on the same atomic unit: a node that tests a feature and routes examples.

How does inference follow conditions to a leaf?

At prediction time, a tree consumes one example at a time. The example enters at the root, evaluates the condition there, and is sent to the matching child. The process repeats at each internal node until the example arrives at a leaf, at which point the leaf's stored value is returned. Google's glossary calls this sequence the inference path ^[1].

For an ensemble, each tree produces its own prediction from its own chain of conditions, and the ensemble aggregates them. Random forests average for regression or vote for classification. Gradient boosted models sum the predictions, often after a sigmoid or softmax for classification. Because inference only evaluates one condition per level, decision forests stay fast even when they contain hundreds or thousands of trees.

Related concepts

Scroll sideways for more →

Many practical hyperparameters in libraries like scikit-learn, XGBoost, and LightGBM (max_depth, min_samples_split, min_child_weight, max_features) are constraints on which conditions the splitter is allowed to consider or how many conditions a tree can contain.

Explain like I'm 5

Think of a decision tree as a game of yes-or-no questions. A condition is one of those questions. You start with one question at the top: "Does the animal have feathers?" If the answer is yes, you go one way and ask another question. If no, you go the other way. You keep answering until you reach the bottom, which is your final guess ("penguin!"). Each question is a condition. The computer learns which questions to ask by looking at a lot of examples and figuring out which question splits them most cleanly into the right groups.

References

1. Google Developers. "Machine Learning Glossary: Decision Forests." https://developers.google.com/machine-learning/glossary/df ↩
2. Google Developers. "Decision Forests: Types of conditions." https://developers.google.com/machine-learning/decision-forests/conditions ↩
3. Google Developers. "Decision Forests: Decision trees." https://developers.google.com/machine-learning/decision-forests/decision-trees ↩
4. Google Developers. "Decision Forests: Course Summary." https://developers.google.com/machine-learning/decision-forests/summary ↩
5. Murthy, S. K., Kasif, S., and Salzberg, S. (1994). "A System for Induction of Oblique Decision Trees." *Journal of Artificial Intelligence Research*, 2, 1-32. https://www.jair.org/index.php/jair/article/view/10121 ↩
6. Google Developers. "Decision Forests: Growing a decision tree." https://developers.google.com/machine-learning/decision-forests/growing ↩
7. Quinlan, J. R. (1986). "Induction of Decision Trees." *Machine Learning*, 1(1), 81-106. ↩
8. Breiman, L., Friedman, J. H., Olshen, R. A., and Stone, C. J. (1984). *Classification and Regression Trees*. Wadsworth. ↩
9. Chen, T., and Guestrin, C. (2016). "XGBoost: A Scalable Tree Boosting System." In *Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining*. ↩
10. scikit-learn documentation. "DecisionTreeClassifier." https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html ↩
11. Tomita, T. M., et al. (2020). "Sparse Projection Oblique Randomer Forests." *Journal of Machine Learning Research*, 21(104), 1-39. https://jmlr.org/papers/v21/18-664.html ↩
12. Hyafil, L., and Rivest, R. L. (1976). "Constructing Optimal Binary Decision Trees is NP-Complete." *Information Processing Letters*, 5(1), 15-17. ↩