The Price for Setting Up the AI Age
These days when ever I wake up multiple companies just reached a $10 Billion valuation, more than five $20 - $50 Billion infrastructure project has been announced, An $80 Billion investment has been committed. If I were waking up from a 5 year comma. I would really think the world has gone crazy coz what the heck are these numbers?
Now to put $80 Billion into perspective, If it were distributed equally among the global population of 8 billion people, every single one of us would receive $10. Ideally this kind of spending would have been ridiculous but a few things Actually validate this new behavior.
1. The fact that Chat GPT was released and in 2 months it reached 100 million people (that's like 0.01% of the earth's population in 2 months which is wild), It meant that AI is a technology that we love and need as humans. For most people, In that moment we could at least imagine to some extent what the future was going to look like with AI in the picture. Everything we knew was about to change.
2. At the moment, very few main stream algorithms could do more than 1 thing. But it turned out that you can use a number of algorithms to train a model to recognize patterns and solve complex problems by learning from data. The model could then be either generalized to do almost anything, or specialized to do a single task. At that moment Anyone who had every written a software program or designed an algorithm before could see that a new Age had started.
3. The sheer number of new things that could now be done was hard to imagine. All programs that existed in the real world before that point were static or preprogrammed so this literally opened up a new world of possibilities. The fact that you could now create a program that could learn from data, adapt to new data and make decision based on that data all in real time was just mind blowing. It meant that A whole lot of new idea, companies and products could now be created.
4. The best thing about this was that the recipes for the creation of these kinds of models are mostly available for free. Science is an open practice after all so a quick google search should get you the knowledge. And it turns out it is actually really easy to create them. The only major problem is that the resource requirement and the scale at which you need to operate in order to implement any model that can be useful or usable for anything is just crazy.
Recommended by LinkedIn
5. You need a lot of data. At least every single word contained on the internet and more. These days you can easily grab yourself a snapshot copy of the entire internet as there are entire organizations dedicated to collecting all the pages on the internet. At least the ones that are collectable and making them available for free.
Now we have the data and have designed the model, The next thing is actually training the model. Now lets say you are Meta and are going to train the llama 3 language model to integrate into your social media platforms. After some research you find that you are going to need 24000 GPUs that cost around $30,000 each. That's $720 million just for the GPUs.
You also need to pay for the electricity to run the GPUs since each on of them uses about 3,740 kilowatt-hours (kWh) when working at 61% for a year, The total electric bill here is also another staggering number but you can calculate based on your area and see what it looks like. We are not done. We also need to install a good cooling system to keep the GPUs from overheating, we need to find space to install the GPUs then 50 - 100 talented people to maintain and utilize the GPUs. As of right now the phrase "YOU NEED A TON OF MONEY FOR THIS PROJECT" is such an understatement.
6. To top this all up, the science shows that the bigger the dataset, the bigger the model and the longer it is trained, the better it will be at the tasks its supposed to do so if you can get, find or create an even bigger dataset than the internet, design a very big model and allow it to train for as long as you possibly can, then you will get a very good model.
The sad thing is that we don't have any more data than the internet and everyone wants to own their AI as seen with the investments made by Google, Microsoft, Amazon, Openai, the Saudi Government etc. Government, companies, institutions, organizations etc. are investing such Huge sums of money to ensure that their AI projects can see the light of day.