Interface administrators, Administrators (Semantic MediaWiki), Curators (Semantic MediaWiki), Editors (Semantic MediaWiki), Suppressors, Administrators
7,785
edits
No edit summary |
m (Text replacement - "Category:Machine learning terms" to "Category:Machine learning terms Category:not updated") |
||
(4 intermediate revisions by the same user not shown) | |||
Line 9: | Line 9: | ||
#[[Mean squared error]] (MSE) Loss: This loss function is the most commonly employed for [[regression]] problems. It measures the average squared difference between predicted output and actual output. | #[[Mean squared error]] (MSE) Loss: This loss function is the most commonly employed for [[regression]] problems. It measures the average squared difference between predicted output and actual output. | ||
#[[Binary cross-entropy]] (BCE): Used in [[binary classification]] problems where the objective is to accurately predict one of two possible [[class]]es, this statistic measures the | #[[Binary cross-entropy]] (BCE): Used in [[binary classification]] problems where the objective is to accurately predict one of two possible [[class]]es, this statistic measures the distance between the predicted probabilities and the actual target values. | ||
#[[Categorical cross-entropy]] (CCE): Used in multiclass classification problems to predict one of several classes, this statistic measures the difference between a predicted probability distribution and an actual one-hot encoded class label. | #[[Categorical cross-entropy]] (CCE): Used in [[multiclass classification]] problems to predict one of several classes, this statistic measures the difference between a [[predicted probability distribution]] and an actual [[one-hot encoded]] class [[label]]. | ||
#Softmax | #[[Softmax cross-entropy]]: This approach is used for multiclass classification problems with mutually exclusive classes. It calculates the [[categorical cross-entropy]] loss for each class and then takes its average across all classes. | ||
#[[KL-divergence]]: This statistic measures the difference in probability distributions. It's commonly employed when training [[generative models]] such as [[Generative Adversarial Network]]s (GANs). | |||
==How Training Loss is Used== | ==How Training Loss is Used== | ||
The training loss is used to evaluate the performance of a machine learning model during training. To minimize this loss, optimization | The training loss is used to evaluate the performance of a machine learning model during training. To minimize this loss, [[optimization algorithm]]s such as [[stochastic gradient descent]] (SGD) or [[Adam]] are employed. These optimization processes modify the model's [[parameters]] in order to minimize its training loss. | ||
==Overfitting and Underfitting== | ==Overfitting and Underfitting== | ||
Overfitting occurs when a model is too complex and fits the training data | [[Overfitting]] occurs when a model is too complex and fits the [[training data]] too well yet fails to generalize well on unseen data. This leads to low training losses but high [[validation loss]]es. [[Regularization]] techniques like [[L1 regularization|L1]] or [[L2 regularization]] can be used as penalties for complex models in order to minimize overfitting. | ||
Underfitting occurs when a model is too simple and does not fit the training data well, leading to high training loss and validation loss. To avoid this issue, more complex models can be utilized or more data can be collected for training the model. | [[Underfitting]] occurs when a model is too simple and does not fit the training data well, leading to high training loss and validation loss. To avoid this issue, more complex models can be utilized or more data can be collected for training the model. | ||
==Explain Like I'm 5 (ELI5)== | ==Explain Like I'm 5 (ELI5)== | ||
Training loss is like a | Training loss is like a score that tells us how well our model is doing at guessing what we're teaching it. When teaching a model, we give it examples and ask it to guess the answer. The training loss is an indicator of how close these guesses are to reality - the lower the loss, the better! Just like when playing video games and getting high scores means you're doing well! | ||
The training loss is | |||
[[Category:Terms]] [[Category:Machine learning terms]] | [[Category:Terms]] [[Category:Machine learning terms]] [[Category:not updated]] |