10 Must-Know Classification Metrics for Machine Learning

• Classification is a supervised learning task in which we try to predict the class or label of a data point based on some feature values. 
• Evaluating a machine learning model is just as important as building it. In this post, we will go over 10 metrics for evaluating the performance of a classification model.
• The metrics we will cover in this article are:

• Classification accuracy

Classification accuracy is a measure of the performance of a classifier, calculated as the number of correct predictions divided by the total number of predictions made. It gives an overall assessment of how well a classifier is able to distinguish between different classes.

Accuracy is a commonly used metric for evaluating the performance of a classifier, but it is not always the most appropriate metric. For example, if there is a class imbalance in the dataset (i.e., one class is much more common than the others), accuracy can be misleading. In such cases, metrics such as precision, recall, and F1 score may be more appropriate.

To calculate accuracy, the following formula is used:

Accuracy = (True Positives + True Negatives) / Total Observations

Where,

True Positives: the number of instances where the model correctly predicted positive outcomes.

True Negatives: the number of instances where the model correctly predicted negative outcomes.

Total Observations: the total number of instances in the dataset.

• Confusion matrix

A confusion matrix is a table that is used to evaluate the accuracy of a classification algorithm. It is a way to visualize the performance of a model in terms of true positive (TP), false positive (FP), true negative (TN), and false negative (FN) predictions.

The matrix consists of 4 quadrants:

True Positive (TP): The number of correct positive predictions made by the model.

False Positive (FP): The number of incorrect positive predictions made by the model.

False Negative (FN): The number of incorrect negative predictions made by the model.

True Negative (TN): The number of correct negative predictions made by the model.

The information in the confusion matrix is used to calculate various metrics, such as accuracy, precision, recall, F1-score, and ROC curve, to assess the performance of the model.

• Precision

Precision refers to the degree of accuracy and reproducibility of a measurement or calculation. It describes how close a set of measurements or results are to each other, and how close they are to the true or expected value. Precision does not necessarily mean accuracy, as a measurement may be precise but still not accurate if it deviates significantly from the true value.

• Recall

Recall refers to the ability to retrieve or remember information that has been previously learned or experienced. It is an essential aspect of memory and helps individuals to retrieve information that is stored in their memory. Recalls can take different forms, such as free recall, cued recall, serial recall, and more. The ability to recall information depends on various factors, such as the type of information, the amount of time that has elapsed since the information was learned, and the individual's level of attention and focus while encoding the information.

• F1 score

F1 score is a metric used to evaluate the performance of binary classification models. It is the harmonic mean of precision and recall and is a balance between the two. Precision measures the proportion of positive predictions that are correct, while recall measures the proportion of actual positive cases that are correctly predicted. The F1 score is a single number that summarizes the accuracy of a model in terms of its ability to identify positive cases and minimize false positives. The F1 score ranges from 0 to 1, with 1 being the best possible score, indicating that the model has perfect precision and recall.

• Log loss

Log loss, also known as logarithmic loss, cross-entropy loss, or negative log-likelihood loss, is a measure of the difference between the predicted probability distribution and the true probability distribution. It is commonly used in classification problems where the aim is to predict the class of an observation based on a set of features.

The lower the log loss, the better the model's predictions match the true probabilities. The log loss is also sensitive to class imbalance, so it's important to consider this when evaluating a model's performance.

The log loss formula is given by:

log loss = -(1/N) * ∑(y * log(p) + (1 - y) * log(1 - p))

Where:

N is the number of observations

y is the actual label of the observation

p is the predicted probability of the positive class for the observation

• Sensitivity

Sensitivity refers to the ability of a system, device, or individual to detect and respond to changes or stimuli in their environment. It can also refer to the degree of responsiveness or responsiveness to subtle changes or influences. In the context of emotions, sensitivity refers to the tendency to easily become affected by emotions or feelings, either one's own or others.

• Specificity

Specificity refers to the degree of exactness or precision of a target or goal in relation to a particular task, requirement, or expectation. It involves specifying the details and parameters of a particular objective or outcome, making it clear and unambiguous. The greater the specificity, the easier it is to understand and measure the success or failure of a particular goal. Specificity helps to ensure that expectations and objectives are aligned, and helps to prevent confusion or misinterpretation.

• ROC curve

A ROC curve is a graphical representation of the performance of a classification model. It stands for the Receiver Operating Characteristic curve and is a plot of the True Positive Rate (TPR) against the False Positive Rate (FPR) at various classification thresholds. The ROC curve is used to evaluate the accuracy of a model in predicting binary class outcomes and to determine the optimal threshold for classifying the positive class. The area under the ROC curve (AUC) is often used as a performance metric, with a value of 1 indicating perfect accuracy and 0.5 indicating random classification. The ROC curve provides a visual representation of the trade-off between the TPR and FPR and allows for the evaluation of a model's performance under different classification thresholds.

• AUC

AUC stands for "Area Under the Curve." It is a measure of the performance of a binary classification model. AUC is a value between 0 and 1, where a higher value indicates better model performance. AUC is commonly used to evaluate the performance of a model by comparing the prediction of the model to the actual class labels. In a perfect model, the AUC would be equal to 1.0, indicating that all positive examples are correctly predicted as positive and all negative examples are correctly predicted as negative.