Kaggle Advanced Regression - Random Forest Without XGBoost
I wanted to experiment on how good (bad) random forest performs at solving both regression and classification problems.
Please check out my Kaggle post that was submitted towards the House Pricing Advanced Regression competition. The model performance in comparison to other submissions came in at the 50th percentile.
In summary, we used One Hot Encoding to deal with categorical values in the model. We also introduced the SimpleImputer to deal with missing values which was very helpful in enriching the quality of data in the model. Last but not least, we used the Random Forest Regressor which seemed to only get half the job done due to its inability to extrapolate outside of the bounds, in simpler terms, if a model has only been trained on a specific range of data, it's risky to use it to make predictions or decisions for situations beyond that range. Just like if you've only studied math up to algebra, you might not be prepared to solve calculus problems. Extrapolation refers to extending or projecting trends or patterns beyond the known data, but this can lead to unreliable results if the model hasn't seen similar situations before.
Recommended by LinkedIn
In the case of a house sale, the same house can get an offer put in up to 50% higher or less of the original sale price depending on the buyers mood which is a very unreliable predictor.
Lastly, the model was missing Gradient Boosting. For example, XGBoost takes bias out of the sensitive features when they are included in the model. With XGBoost added after the random forest regressor, the model easily makes it to the top 10% submissions.
I hope you can pick something from this in your next assignment. Thanks for reading and please join the community at Kaggle to test your might :)