Interface administrators, Administrators (Semantic MediaWiki), Curators (Semantic MediaWiki), Editors (Semantic MediaWiki), Suppressors, Administrators
7,785
edits
Backpropagation: Difference between revisions
no edit summary
No edit summary |
No edit summary |
||
| Line 1: | Line 1: | ||
==Introduction== | ==Introduction== | ||
Backpropagation is a technique used to train artificial neural | [[Backpropagation]] is a technique used to train [[artificial neural network]]s, [[machine learning models]] inspired by the structure and function of the human brain. This process adjusts the weights assigned to connections within the network in order to minimize [[error]]s between [[prediction]]s and actual [[input]] values. | ||
The backpropagation algorithm is often employed in conjunction with supervised learning tasks, in which the network is provided a set of input-output pairs and asked to learn a mapping from inputs to outputs. The algorithm consists of two primary steps: forward propagation and backward propagation. | The backpropagation algorithm is often employed in conjunction with [[supervised learning]] tasks, in which the network is provided a set of input-output pairs and asked to learn a mapping from inputs to outputs. The algorithm consists of two primary steps: [[forward propagation]] and [[backward propagation]]. | ||
==Forward Propagation== | ==Forward Propagation== | ||
In the forward propagation step, input data is routed through a network, and output is calculated by applying an activation function to the weighted sum of inputs at each neuron. An activation function is an algebraic formula that determines a neuron's output given its inputs; common activation functions include | In the forward propagation step, input data is routed through a network, and output is calculated by applying an [[activation function]] to the weighted sum of inputs at each neuron. An activation function is an algebraic formula that determines a neuron's output given its inputs; common activation functions include [[Sigmoid]], [[ReLU]] and [[Tanh]]. | ||
==Backward Propagation== | ==Backward Propagation== | ||
In the backward propagation step, the weights of the network are adjusted. The aim is to minimize any error between predictions and actual values by using a loss function such as mean squared error or cross-entropy. | In the backward propagation step, the [[weights]] of the network are adjusted. The aim is to minimize any error between predictions and actual values by using a [[loss function]] such as [[mean squared error]] or [[cross-entropy]]. | ||
Calculating the gradient of a loss function with respect to weights requires using the chain rule of calculus. This rule states that to obtain the derivative of a composite function, one must multiply its derivative by that of its inner function. In backpropagation, however, we assume that the outer function is the loss function and its inner one represents activation levels for neurons. | Calculating the [[gradient]] of a loss function with respect to weights requires using the chain rule of calculus. This rule states that to obtain the derivative of a composite function, one must multiply its derivative by that of its inner function. In backpropagation, however, we assume that the outer function is the loss function and its inner one represents activation levels for neurons. | ||
To adjust the weights on a network, an optimization algorithm such as gradient descent is employed; here, the weights are updated in the opposite direction to that of the loss function's gradient. This process is repeated multiple times until there is a minimal error between predictions and true values. | To adjust the weights on a network, an [[optimization algorithm]] such as [[gradient descent]] is employed; here, the weights are updated in the opposite direction to that of the loss function's gradient. This process is repeated multiple times until there is a minimal error between predictions and true values. | ||
==Explain Like I'm 5 (ELI5)== | ==Explain Like I'm 5 (ELI5)== | ||
Backpropagation is a technique used by computers to learn new tasks, like recognizing pictures or understanding speech. This process helps the computer determine whether it got an answer correct or incorrect and makes adjustments so it can do better next time around - just like when teachers mark tests and tell you what went wrong and how you can improve for future attempts. Backpropagation allows computers to learn on their own. | Backpropagation is a technique used by computers to learn new tasks, like recognizing pictures or understanding speech. This process helps the computer determine whether it got an answer correct or incorrect and makes adjustments so it can do better next time around - just like when teachers mark tests and tell you what went wrong and how you can improve for future attempts. Backpropagation allows computers to learn on their own. | ||
[[Category:Machine learning terms]] [[Category:Terms]] | |||