Bias

Introduction

Bias in mathematics and machine learning refers to the difference between an estimator's expected value and the true value of a parameter being estimated. In other words, bias introduces systematic error into an estimation process.

Machine learning often refers to bias when discussing supervised learning algorithms. Supervised learning algorithms are trained on a dataset composed of input-output pairs with the purpose of discovering an mapping between inputs and outputs. The bias of such an algorithm serves as a measure of its capacity for fitting training data accurately.

Bias-Variance Tradeoff

The bias-variance tradeoff is a fundamental concept in the analysis of supervised learning algorithms. According to this tradeoff, an algorithm with high bias will have low variance and vice versa; that is, an algorithm with high bias won't be as sensitive to specific training data provided but also less likely to fit it well. On the other hand, one with low bias will be more sensitive to specific training data given but also more likely to fit it perfectly.

Bias in Neural Networks

Neural networks use a bias term, which is an integer value added to each neuron's weighted sum of inputs. This bias allows the output of the neuron to be shifted up or down and plays an essential role in how well a neural network fits various input-output mappings. The bias term can be learned during training and typically starts as small random values.

Explain Like I'm 5 (ELI5)

Bias is like a cheat code in a video game; if it's too hard, you can use one to make it simpler. But if you use too many cheat codes, you might not learn how to play effectively. In machine learning, bias helps make it simpler for the computer to find correct answers; however, too much bias could prevent the machine from learning how to find those answers on its own. A bias term is simply an extra number added onto each math equation to assist the machine in finding its correct response.