Attribute

See also: Machine learning terms

In machine learning and data mining, an attribute is an individual measurable property of an object, observation, or example that describes the instance being analyzed, and it forms one column of a tabular dataset. ^[1] Attributes supply the raw signal a model uses to learn patterns or make predictions, and a single row of values across all of an instance's attributes is what a model takes as input. The term is largely synonymous with feature, though some texts draw a fine distinction in which an attribute is a quantity describing an instance and a feature is the specification of an attribute together with a concrete value. ^[2]

Introduction

In machine learning, an attribute is an individual measurable property of an object, observation, or example that describes the instance being analyzed. Attributes are the columns of a tabular dataset, and they supply the raw signal that a model uses to learn patterns or make predictions. In a table of patient records, age, blood pressure, gender, and cholesterol level are attributes; in a table of houses, the number of bedrooms, square footage, neighborhood, and year built are attributes.

The word attribute is most common in the data mining and database traditions, where a column of a relational table is called an attribute. The same idea appears in pattern recognition and machine learning under the name feature, and in classical statistics under names such as independent variable, predictor, regressor, explanatory variable, or covariate. These terms are mostly interchangeable in practice, though each community has its own house style.

What is the difference between an attribute and a feature?

The two words attribute and feature are often used as synonyms, but a careful distinction does exist. In the glossary of terms published by Ronny Kohavi and Foster Provost in the journal Machine Learning in 1998, an attribute is defined as "A quantity describing an instance," while a feature is defined as "The specification of an attribute and its value." ^[2] The same glossary gives the textbook illustration: "color is an attribute. 'Color is blue' is a feature of an example." ^[2] Under that definition, the attribute is the slot; the feature is the slot together with a concrete reading.

This subtlety matters in older data mining literature and in some textbooks, but it has largely faded in everyday usage. Most modern documentation, including the Wikipedia article on features, treats feature and attribute as the same thing. That article defines a feature, following Christopher Bishop's 2006 textbook Pattern Recognition and Machine Learning, as "an individual measurable property or characteristic of a data set." ^[1] The Domino Data Lab data science dictionary also lists variable and attribute as synonyms for feature. ^[7] The practical result is that a data scientist who talks about "adding a feature" and a database analyst who talks about "adding an attribute" are usually doing the same job.

There is also a small terminology drift across subfields. Computer vision and signal processing tend to favor feature, especially when referring to the output of a transformation such as a feature extraction step or an edge detector. Tabular data work, business intelligence, and survey research lean toward attribute or variable. None of these labels change what the underlying object is.

Attributes as independent variables

In supervised learning, attributes act as the independent variables of the model. A predictive system takes a vector of attribute values as input and returns an estimate of a label or target value. The label is the dependent variable; the attributes are everything used to predict it. The Wikipedia article on dependent and independent variables lists feature and label attribute as the data mining equivalents of independent and dependent variables in statistics. ^[6]

This framing also clarifies a common naming question. A target column in a training data table is technically still an attribute of the row, but it is a special one: the value the model is trying to learn. To avoid confusion, many practitioners reserve the bare word attribute for the inputs and call the output the target, label, or response.

What types of attributes are there?

Attributes come in several flavors, and the type determines what kinds of mathematical operations make sense on the values. The data mining community usually divides attribute types along two axes: by data nature and by level of measurement.

By data nature, the most common split is:

Scroll sideways for more →

Numerical attributes are further split into discrete (integer counts such as number of children) and continuous (real-valued such as temperature). Categorical attributes are further split into nominal, where no order exists between categories, and ordinal, where a meaningful order does exist.

By level of measurement, attributes are usually grouped using the classic scheme that the psychologist Stanley Smith Stevens proposed in a 1946 paper in the journal Science titled "On the Theory of Scales of Measurement": nominal, ordinal, interval, and ratio. ^[3] Stevens argued that the mathematical operations that are permissible on a set of numbers depend on the scale of measurement used to produce them, and the four scales form a hierarchy in which each level keeps every property of the level below it and adds one new property. ^[3] Nominal attributes only support equality checks; zip code and eye color are typical examples. Ordinal attributes also support ordering; a Likert scale of {poor, fair, good, very good, excellent} is ordinal. Interval attributes support meaningful differences but lack a true zero; Celsius and Fahrenheit temperatures are the textbook cases. Ratio attributes support meaningful ratios and have a real zero point, like weight, height, or counts.

The distinction is not academic. A linear model that treats a zip code as a number assumes that the gap between 10001 and 10002 means the same thing as the gap between 90000 and 90001, which is obviously wrong. Knowing the measurement level of every attribute is the first step in choosing a sensible preprocessing pipeline.

How are attributes preprocessed before training?

Most learning algorithms expect numeric input, so raw attribute values usually need transformation before training. Common preprocessing steps include:

Scroll sideways for more →

Tree-based methods such as random forests and gradient boosting can handle some categorical attributes directly, but most other algorithms cannot. One-hot encoding is the default trick for nominal data: a single attribute with three values, like color in {red, green, blue}, becomes three binary columns. Standardization is the default for numeric data when the model cares about scale, which includes linear regression, logistic regression, support vector machines, k-nearest neighbors, and most neural networks.

This whole step is often called feature engineering. ^[4] The same column on disk can become several different attributes in a trained model: a date might end up as year, month, day of week, and an is_weekend flag, and a price might end up as a raw value, a log-transformed value, and a percentile rank. The transformations are part of the model, not metadata about the data.

Why does attribute choice matter?

Attributes are how the world enters the model. If the attributes do not contain the signal needed to predict the target, no amount of clever training will recover it. A spam classifier without any text features is not going to work, no matter which algorithm you pick.

At the same time, more attributes is not automatically better. Adding irrelevant or redundant attributes makes models harder to interpret, slower to train, and more prone to overfitting. It can also trigger the curse of dimensionality, a term the mathematician Richard E. Bellman coined in his 1957 book Dynamic Programming while working on dynamic programming. ^[5] As the number of attributes grows, the volume of the input space grows exponentially, the data becomes sparse, and distances between points lose their meaning. ^[5] Many algorithms degrade badly in this regime.

The usual response is feature selection: pick a subset of attributes that carry most of the useful signal and drop the rest. ^[9] There are three main families of feature selection methods:

Scroll sideways for more →

A related response is dimensionality reduction, where new attributes are constructed as combinations of the originals. Principal component analysis is the textbook example: it builds a small set of new attributes that capture as much variance as possible from the full attribute set. Other techniques include linear discriminant analysis, independent component analysis, and autoencoders.

Attribute vectors and attribute space

A single row of a dataset, with all of its attribute values filled in, is sometimes called a feature vector. The set of all possible feature vectors for a given attribute list is the feature space. A model can then be described geometrically: a classification model carves the feature space into regions for each class, and a regression model draws a surface over it. This geometric view is what makes the curse of dimensionality bite. Adding attributes means stretching the space into more dimensions, and the data points that used to feel close together start to look uniformly far apart.

A worked example

Consider a small dataset for predicting whether a customer will churn from a streaming service. The raw table might have these attributes:

Scroll sideways for more →

A practitioner would drop customer_id, one-hot encode country (or use a hashing trick if there are thousands of values), label encode plan_type while preserving its order, log-transform monthly_spend, derive year, month, and tenure_months from signup_date, and keep age and last_login_days as is. The result is a clean attribute matrix that any standard machine learning algorithm can consume.

The choice of attributes and the way they are transformed often matters more than the choice of model. Two teams using the same gradient boosting library on the same raw data can produce wildly different results because one team built better features.

Explain like I'm 5

Imagine you are trying to guess how much a house costs. You would not just look at a picture and shrug. You would write down some facts: how many bedrooms it has, how big it is, what neighborhood it is in, how old it is. Each fact is an attribute. The computer does the same thing. It looks at lots of houses and their attributes, learns which facts matter, and then uses those facts to guess the price of a new house it has never seen.

Some attributes are numbers, like square footage. Some are words, like neighborhood names. The computer prefers numbers, so we usually translate the words into numbers first. Once everything is in a friendly shape, the computer can do its job.

References

1. Wikipedia, Feature (machine learning), https://en.wikipedia.org/wiki/Feature_(machine_learning) (definition attributed to Christopher Bishop, Pattern Recognition and Machine Learning, Springer, 2006). ↩
2. Ronny Kohavi and Foster Provost, Glossary of Terms, Machine Learning, vol. 30, pp. 271-274, 1998, http://robotics.stanford.edu/~ronnyk/glossary.html ↩
3. S. S. Stevens, On the Theory of Scales of Measurement, Science, vol. 103, no. 2684, pp. 677-680, 1946, https://en.wikipedia.org/wiki/Level_of_measurement ↩
4. Wikipedia, Feature engineering, https://en.wikipedia.org/wiki/Feature_engineering ↩
5. Richard E. Bellman, Dynamic Programming, Princeton University Press, 1957; see Wikipedia, Curse of dimensionality, https://en.wikipedia.org/wiki/Curse_of_dimensionality ↩
6. Wikipedia, Dependent and independent variables, https://en.wikipedia.org/wiki/Dependent_and_independent_variables ↩
7. Domino Data Lab Data Science Dictionary, Feature, https://domino.ai/data-science-dictionary/feature ↩
8. Deepchecks Glossary, Attribute, https://deepchecks.com/glossary/attribute/
9. Wikipedia, Feature selection, https://en.wikipedia.org/wiki/Feature_selection ↩
10. Sebastian Raschka, Filter, wrapper, and embedded methods for feature selection, https://sebastianraschka.com/faq/docs/feature_sele_categories.html
11. dotData, Feature Engineering for Categorical Attributes, https://dotdata.com/blog/feature-engineering-for-categorical-attributes/