Why do projects get delayed?

HPC, AI, ML, Generative AI, Quantum Computing have accelerated much faster than would have been expected 5 years ago.  So, what does that mean to all of us? Are we ready to journey down this path?

The first thing you must know is that these new directions are going to require more powerful technology.  This technology will be chosen based on the resources needed specific to the path chosen.  But what is forgotten is that one resource requires another resource to function.  And that one requires another resource to function. So on and so on until you have stretched your resources too thin.  Planning for these is more than just the compute power required or the GPUs needed or even the amount of storage required.  Most of you understand this and know this but does the planning really happen the way we know it should.  No, it does not.  Most projects take months and there are delays for unseen circumstances.  This is due to the planning of a project getting bogged down by the natural flow of communication within a company or organization.  Next is underestimating the technology needed for this project.  Lastly, not calculating the resources required vs what is available not only in the physical building but in the area around the building.

Planning is crucial but often it happens in stages.  New technology path is chosen and there is promise in making a successful journey down the path.  This gets approved by a leader, typically the CIO.  The CTO or IT Director is tasked to find the solution to make this happen.  They call a few vendors they work with and find the options for the technology that supports the new path chosen.  Once validated it usually finds its way to the networking admin to make sure it can connect and get the bandwidth needed.  After evaluating the needs for the network, this project then moves to the Datacenter admin.  That person will be tasked to find a home for this.  Where will it go in the datacenter?  Will it fit and what will need to be moved to accommodate this new technology.  You would think that this is the end of it.  We have everything we need to make the decision and sometimes that happens.  If the Datacenter admin or manager is closely working with their facilities people, they will know to check with them if there is enough power to supply this project.  If they move or shift or add to any of these circuits, will it overload that circuit.  This timeline to hit all of these can typically take more than a month if not multiple months.  I have seen this and been part of these large projects with my piece of the puzzle.  It happens more than you think.  So, what is the solution?  Planning should be as important as board meetings and structured similarly.  Everyone that has a part in the total solution should come to the table all at once and review all the details.  This streamlines the process and reduces the time to decide.  You can comment all you want on how you see this happen but, this process of planning a project is very broken in most cases.  Emails are sent and awaiting response, but the next person does the same thing, email sent and awaiting response.  It can take weeks to get one answer out of many answers needed.  Take the time to evaluate the goals of this project, timelines for delivery, needs for adding or distributing resources and make all these meetings together. 

Why do we have roadblocks or delays in deployments?  Underestimating the resources that the technology requires to complete the goals.  The server is too big, has a different configuration, requires more power, requires higher bandwidth and many more.  What about the network?  Did they plan for the need for more bandwidth or more connections?  Can the core handle this new workload or will it need to be upgraded at the last minute, further delaying the project’s deployment.  Do you have all the cables needed to connect to the network?  Do you have enough storage? How much more is needed and how much space is that going to take up? Is it going to fit in the existing datacenter? Great, all that was thought of, and it is now deployed in the datacenter.  Did anyone make sure there was enough power, or this was not going to overload a circuit?  What about the increased heat generated?  Did cooling come into the conversation or were assumptions made that there was capability to properly power and cool this project.  Many of these happen way more than most think.  It delays projects deployment; people start blaming others for not considering this and often adds more cost to this than previously considered.  Everyone in the organization that potentially has a connection to this project needs to be part of the planning process.  My part in the process must deal with the resources and the infrastructure needed to supply these resources.  So, I see projects come to a halt all the time for missing cables, power cords, underestimating the power on each circuit as well as the ability to keep it cool.  Adding resources is always an option but what some fail to do is check to see if they can increase resources while not affecting the community around them.  If they increase cooling, power draw and circuit size to accommodate more technology, what strain are they putting on the power grid or water supply in the area.  You can plan for all the resources you need but if the local supply for these resources cannot support you, then your project is on hold or scrapped.  Power grids are getting constrained more and more with all the growth.  Water supply is also a topic of debate for communities.  You may increase demand and constrain the community creating outages and limitations for usage.  This will also affect your ability to fully leverage the new technology you have increased demand for.

In most cases, communication is broken down.  Working with manufacturers and vendors in the IT Technology business, I see it daily.  Leverage your partnerships but also be open to sharing what is needed for the full project.  Many vendors or manufacturers have solutions or partner with those who have solutions to fit the needs of this project and your organization.  If you are starting a new path in advanced technology, bring everyone to the table.  Vendors, Manufacturers, directors, admins, facilities as well as the local providers of your resources.  There are new technologies not only in power but cooling that can help minimize the demand for resources.  There is not going to be one solution but multiple solutions to support your project when it comes to the resources needed. 

There have been many changes post Covid regarding communication and planning in organizations.  People are remote and often in different time zones. Some have never even been in the datacenter and have no idea what will be needed for this project. We need to see there are gaps and make the changes to bring these gaps together.  You do not always need to hire a consultant or project manager.  You just need to change the culture around your communication within your organization.  Then you will see an increase in productivity as well as faster deployments of projects.  A few extra weeks could affect the success of the project and how far you take it down this new path.