Risks around AI implementations

As the buzz around AI peaks, it can become difficult to separate the hype from the substance and value. I am seeing participation in conferences around AI multiply, and a lot of it is fuelled by the fear of being left behind. While that fear is justified to some extent, it can get to a stage of paralysis where teams move from one thing to the next without generating real value for their organisations. In my work at Spark New Zealand and in my role at the executive council at AI Forum NZ , I have tended to index towards the minimum viable proposition that creates some tangible value for the organisation.

During a recent panel discussion, I was asked a question  about risk management frameworks that need to be in place to empower your tech and data teams to embrace AI initiatives.  I thought that I would post my response to the question, as it might be of help to those in Aotearoa, who are thinking of being on the journey of AI, or have initiated the journey recently.

While I was writing this, I started using Google's NotebookLM, and while it's not perfect like many things in the AI space, but it's a better podcast that I could have created after spending hours. This took me roughly 30 mins to put together, the input content I provided, was this article without any additional prompts.

Podcast created using NotebookLM

There are a few different types of risks around AI initiatives and projects.

1. The risk that the team is not able to create a functional product
2. The risk that the product that is created does not have the right set of guardrails around it
3. The risk that the product that is created is not used by the intended users

Note that there can be other risks associated with scaling which I have left out of this for now.

Lets evaluate each of these risks one by one by the life cycle and maturity stages..

Creation Risk - The risk that the team is not able to create a functional product - This can happen because of several reasons, it can be the question of not having the right infrastructure or  that the data teams do not have the right skills. If it is about infrastructure, it would be good to start off with development environments that are available from the hyper scalers and get something up and running relatively quickly.  The question of skills is important to address, as within the realm of data, certain skills becoming more relevant to the creation of Data Products. In your own teams, there will of course be some who will upskill beyond traditional data engineering, analytics and software engineering, but everyone may not. This is a really good time to understand which skills you need to over index on and be in market for these skills. Considering the current market conditions, where there is surplus supply in market, it's a great time to bring such people on board. Even with the right infrastructure and skills, there is the risk of being caught in an endless loop of POCs and not able to deliver a real outcome for the business. Make sure that the focus is on creating something that delivers business value, rather than moving on to the next shiny thing on the horizon and do the next POC.

Governance Risk - The risk that the product that is created does not have the right set of guardrails around it -  While there is a lot of discussion around governance, this only comes after you have something around which you need governance. There can be simple applications with human in the loop where the degree of governance required will be quite basic. Examples of such applications are creation of first drafts of a marketing copy, which will go through a human review cycle before reaching the next step. I see a lot of activity and discussions around governance, and while this is really important for certain types of applications, this is not the primary thing  one needs to worry about on day one. Having said that, it is important to put in governance practices to mitigate some of behaviours of LLMs. This is where LLM Operations and LLM Evaluation frameworks becomes really important. LLM eval frameworks help to understand the performance of the LLMs against metrics like relevance, hallucination, latency and other success measures that can be created. During the first few projects, I would recommend to go with human led evaluation e.g. human labelling of the outputs, however, as the number of projects increase, and / or the user base expands, it will be difficult to manage these without tools and LLM Ops tools should become a part of the stack at this stage.

Adoption Risk - The risk that the product that is created is not used by the intended users - There needs to be concerted focus on adoption, ideally this should be driven by senior management, leadership squads and CEOs. 30-40% of the effort in an AI initiative needs to go towards adoption of these tools, and ideally this should become a company wide initiative with human resources and people managers playing a key role in encouraging adoption. Effective change management should include a combination of approaches. A natural trend that I see is that, on average, employees who gravitate towards these new technologies sell more or get better NPS scores, and this probably happens because they are naturally more inquisitive and eager to learn and naturally are better performers. A subset of this group are people who are new to the organisation. Eager newcomers who find themselves at an information disadvantage compared to others, find that these tools help them get up to speed much faster, by bringing information to their fingertips. It's good to promote these stories of success of people using these tools and how well they are performing compared to those not using them. A parallel stream of effort needs to go towards using learning modules and constant reminders to complete learning modules so employees can get on to using the tools effectively.

I would love to hear your thoughts and comments, specially on the podcast and stories about where you are on the journey and you are facing these risks or if there are other types of risks and how you are mitigating these.