How can you differentiate common regression evaluation metrics?

MAE is the average of the absolute differences between predicted and actual values. Think of it as an average error score: the higher the MAE, the more the model is off in its predictions, while a lower MAE suggests better accuracy. Advantage of MAE: MAE is straightforward and easy to understand because it simply averages the errors. When you see the MAE value, it directly tells you, on average, how much your predictions missed the mark. Disadvantage of MAE: It treats all errors the same, so a huge error is seen as just as bad as a small one. This means if your model makes one really big mistake, MAE won't emphasize it much, which can be misleading if those big errors are crucial to avoid.

When to Use MAE: 1. When you want an easily interpretable metric. 2. When you want to treat all errors equally, regardless of their direction. 3. When your distribution has long tails or you have outliers that you don't want to remove but also don't want to give high importance. Limitations: 1. It does not penalize large errors as much as metrics like Mean Squared Error (MSE). 2. It may not be suitable for applications where you want to pay more attention to large errors.

It is the mean of the absolute of errors. It is similar to RMSE in the sense that its values are on the same scale as the dependent variable and easier to explain to a layman. It, however, cannot help when we want to suppress large error values and can be biased.

Regression evaluation metrics quantify prediction errors. Key metrics include: 1. Mean Absolute Error (MAE): Average of absolute differences between predicted and actual values. 2. Mean Squared Error (MSE): Average of squared differences, emphasizing larger errors. 3. Root Mean Squared Error (RMSE): Square root of MSE, representing prediction error in original units. 4. R-squared: Proportion of variance explained by the model, indicating goodness of fit. 5. Adjusted R-squared: Like R-squared but accounts for the number of predictors in the model. Differentiate by their sensitivity to outliers (MSE is more sensitive) and interpretation (R-squared values range 0-1).

When to Use MSE: 1.When you want to give more weight to larger errors. 2.When your data is normally distributed or when the distribution of errors is Gaussian. 3.When you are using optimization algorithms that benefit from a differentiable loss function. Limitations: 1. Sensitive to Outliers: A single very large error can dominate the MSE. 2. Scale: The scale of MSE can sometimes make it hard to interpret directly. 3. Human Perception: Squaring the errors can sometimes yield results that don't align with human perception of error.

It is calculated as mean of the square of individual errors. It is easier to calculate than RMSE. It suffers from one drawback, i.e. since the errors are squared before they are averaged, the MSE gives a relatively high weight to large errors. It may lead to biased results if there are large error values.

When to Use RMSE: 1.When you want a metric that is in the same unit as your target variable. 2.When you want to give more weight to larger errors but not as much as MSE. 3.When you are using optimization algorithms that benefit from a differentiable loss function. Limitations: 1.Sensitive to Outliers: While less sensitive than MSE, RMSE can still be influenced by outliers. 2.Not Ideal for All Distributions: RMSE assumes a Gaussian distribution of errors, which may not be the case for all datasets. 3.Variability of Errors: RMSE may not capture the variability of errors across different levels of the dependent variable.

It helps us make the decision if the model is practically useful or not as we can decide from the error if it is acceptable or not. It takes the root of the MSE and hence is not biased towards large values. It gives errors in the same scale of measurement in which dependent variable is measured. Hence it is also easier to explain to layman stakeholders.

When to Use R2 : 1.When you want to measure how well your model explains the variability in the data. 2.When you want a unitless measure to compare different models or datasets. 3.When you are more interested in the goodness of fit rather than the size or direction of errors. Limitations: 1. Overfitting: A high R2 does not necessarily mean a good model; it could be overfitting the data. 2. Not for Comparing Different Types of Problems: Since it's unitless, it might be tempting to compare values across different types of problems, which can be misleading. 3.Not Always Applicable: R2 is most meaningful for linear regression models and may not be as useful for non-linear models.

R square tells us about how much of variation in data is explained by the model. It is an indicator of model performance on the training data. It is not an explanation of how well the model can predict in the real world scenarios, or at least on the test and evaluation data.

R-squared can be interpreted as the percentage of the variation in the dependent variable that can be accounted for by the independent variables. For example, an R-squared value of 0.75 means that 75% of the variability in the dependent variable can be explained by the independent variables, while the remaining 25% is due to other factors or unexplained errors. Therefore, R-squared should not be used as the sole criteria for evaluating the quality of a regression model. Other metrics, such as the standard error of the estimate or the mean squared error, should be considered alongside R-squared to assess the model's overall performance.

The best metric for regression evaluation is the metric in which dependent variable is measured. For example, if commodity price is dependent variable, $ value can be used as metric. Metrics such as RMSE and MAE are in the same scale of measurement, as that of dependent variable. Hence, it makes sense to check how much is MAE for the model on test and validation data, instead of solely relying on R square.