Improving Forecasting Performance in Machine Learning: Don't Give Up!

A personal journey of perseverance, creativity, and systematic improvements in machine learning forecasting.

Every machine learning project is a test of creativity, patience, and persistence. Here’s what I learned while chasing better forecasting performance — and why you should never give up.

When working with machine learning for forecasting tasks, the goal is always to build a model that achieves better performance. One key evaluation metric is Mean Absolute Error (MAE). To make progress measurable, it's important to define a baseline or target MAE. The lower the MAE, the closer the model’s forecast is to the actual values — and the better the model performance.

However, achieving a lower MAE is rarely easy. The challenge arises when the target is not immediately reached, and it becomes a test of the ML engineer's creativity and persistence to explore better methods through experimentation. But this is exactly where the excitement lies — the process of improving the model should be systematic: whether by adjusting hyperparameters, searching for the best configurations through tuning, or innovating new ideas.

The key principle is: don't give up until you absolutely must.

As an example, here’s something I’ve been working on during my spare time outside of office hours — a research journey to achieve “better performance” (or hopefully “best performance”) in a forecasting problem. The illustration below shows the stages of model improvement, tracking how systematic changes led to progressively better results. So far, the improvements are promising, especially in the most recent stages.

This journey isn't over yet — I still have more ideas in mind for the next stages and hopefully the time to share updates here.

If you’re a machine learning engineer working on your own challenges, remember: "Don't give up."

The starting point should always be straightforward: pick any approach or technique that seems reasonable based on your understanding.

From there, the real journey begins — improving the model through structured stages, guided by both logic and experimentation.

Below are some of the improvement ideas that we logically believed could enhance performance. Some worked, some didn’t — but that's the essence of research, isn’t it? It’s about exploring, validating, and constantly learning along the way.

Every improvement — whether big or small — adds another step forward in the journey toward better forecasting.

To capture this process, the following illustration summarizes the key stages of model improvement, highlighting the logical steps taken to enhance performance over time.

As with any research journey, not every attempt brings immediate success. Our latest stage, which we initially believed would lead to further improvement, did not meet the performance targets we had hoped for.

But that’s the beauty of research — it’s not just about the results; it’s about the discovery, the surprises, and the persistence to keep moving forward.

The journey isn’t over yet. The excitement continues — and that's exactly what makes research truly rewarding.

#MachineLearning #Forecasting #DataScience #MLEngineering #ModelOptimization #Persistence #ResearchJourney #ContinuousImprovement