Interface administrators, Administrators (Semantic MediaWiki), Curators (Semantic MediaWiki), Editors (Semantic MediaWiki), Suppressors, Administrators
7,785
edits
Backpropagation: Difference between revisions
no edit summary
Tag: Undo |
No edit summary |
||
| Line 2: | Line 2: | ||
Backpropagation is a technique used to train artificial neural networks, machine learning models inspired by the structure and function of the human brain. This process adjusts the weights assigned to connections within the network in order to minimize errors between predictions and actual input values. | Backpropagation is a technique used to train artificial neural networks, machine learning models inspired by the structure and function of the human brain. This process adjusts the weights assigned to connections within the network in order to minimize errors between predictions and actual input values. | ||
The backpropagation algorithm is often employed in conjunction with supervised learning tasks, in which the network is provided a set of input-output pairs and asked to learn | The backpropagation algorithm is often employed in conjunction with supervised learning tasks, in which the network is provided a set of input-output pairs and asked to learn a mapping from inputs to outputs. The algorithm consists of two primary steps: forward propagation and backward propagation. | ||
==Forward Propagation== | ==Forward Propagation== | ||
In the forward propagation step, input data is routed through a network and output calculated by applying an activation function to the weighted sum of inputs at each neuron. An activation function is an algebraic formula that determines a neuron's output given its inputs; common activation functions include sigmoid, ReLU and tanh. | In the forward propagation step, input data is routed through a network, and output is calculated by applying an activation function to the weighted sum of inputs at each neuron. An activation function is an algebraic formula that determines a neuron's output given its inputs; common activation functions include sigmoid, ReLU and tanh. | ||
==Backward Propagation== | ==Backward Propagation== | ||
| Line 12: | Line 12: | ||
Calculating the gradient of a loss function with respect to weights requires using the chain rule of calculus. This rule states that to obtain the derivative of a composite function, one must multiply its derivative by that of its inner function. In backpropagation, however, we assume that the outer function is the loss function and its inner one represents activation levels for neurons. | Calculating the gradient of a loss function with respect to weights requires using the chain rule of calculus. This rule states that to obtain the derivative of a composite function, one must multiply its derivative by that of its inner function. In backpropagation, however, we assume that the outer function is the loss function and its inner one represents activation levels for neurons. | ||
To adjust the weights on a network, an optimization algorithm such as gradient descent is employed; here, the weights are updated in the opposite direction to that of the loss function's gradient. This process is repeated multiple times until there is minimal error between predictions and true values. | To adjust the weights on a network, an optimization algorithm such as gradient descent is employed; here, the weights are updated in the opposite direction to that of the loss function's gradient. This process is repeated multiple times until there is a minimal error between predictions and true values. | ||
==Explain Like I'm 5 (ELI5)== | ==Explain Like I'm 5 (ELI5)== | ||
Backpropagation is a technique used by computers to learn new tasks, like | Backpropagation is a technique used by computers to learn new tasks, like recognizing pictures or understanding speech. This process helps the computer determine whether it got an answer correct or incorrect and makes adjustments so it can do better next time around - just like when teachers mark tests and tell you what went wrong and how you can improve for future attempts. Backpropagation allows computers to learn on their own. | ||