AI Governance: Why Traditional Oversight Falls Short and the Case for a New Approach in Today’s Evolving Landscape

In recent years, artificial intelligence (AI) has shifted from a novel technology to a core component of business operations, bringing with it unprecedented risks that traditional governance frameworks cannot fully address. While boards have relied on well-established frameworks to manage data security, privacy, and compliance, these approaches fall short when it comes to AI’s unique and complex challenges.

This is driven by three main factors:

• AI introduces novel risks,
• new legal requirements must be integrated into the tech stack to address these risks,
• and specialized skills, processes, and tools are essential for effective management.

1. AI: Not Just Software, But a New Frontier of Novel Risks

AI introduces specific challenges that legacy governance simply isn’t designed to address. 

Core to this difference is that AI differs fundamentally from traditional software due of AI’s ability to learn, adapt, and make decisions based on data, which makes it inherently less predictable than traditional, rule-based software (see NIST AI RMF, Appendix B).

Depending on the system's complexity - from traditional machine learning models like decision trees to intricate, multi-agent systems - these risks become increasingly complex to detect and address. AI systems may exhibit bias, lack transparency, or produce misinformation and unexpected outcomes - risks that traditional models of oversight don’t anticipate. 

The AI Incident Database tracks critical AI risks, including bias and discrimination across demographic factors, sector-specific failures (e.g., in healthcare, law enforcement), technical issues like generalization errors and misinformation generation, as well as operational risks.

In August and September 2024, the AIID added 46 new incidents, including Facebook’s wildfire alert error, Google’s misleading parental advice, extortion and impersonification attempts of high profile leaders, a global escalation in deep fake scams in general, and an AI transcription service inadvertently sending out an AI-generated transcript of private conversations from a VC meeting.

From data security breaches and regulatory fines to reputational damage, each of these cases emphasizes the need for an AI-specific governance framework that includes oversight, quality control, AI security and safety measures, and compliance updates tailored to the unique challenges AI presents. 

2. Following in the Footsteps of Privacy and Security: The Need for Embedded Compliance

AI governance is following a similar path to that of privacy and security, both of which had to fight for recognition as critical, organization-wide concerns. Just as privacy and security ultimately proved their relevance and necessity, AI governance now faces similar challenges in gaining recognition as a company-wide risk area. 

In addition to that, privacy and security have shown that simply having policies is not enough; legal requirements now demand that security and privacy measures be technically embedded into IT systems, products, and infrastructure from the outset - a proactive approach known as "shift left." This practice, along with security and privacy engineering, ensures that these protections are integral to the design and function of technology rather than retrofitted after development. 

The same is true for AI, as AI risk management is now mandated by a growing number of international laws such as the EU AI Act and U.S. state laws (e.g., in Utah, Colorado, and California) and must be directly integrated into the technical architecture of AI systems.

For example, California’s AB 1008 extends existing privacy protections to generative AI systems. CA AB 2013 mandates transparency regarding the data used for training AI models, pushing companies to incorporate data governance practices directly into their technical stacks. Similarly, risk assessments mandated by SB 896 signal the need for AI systems to be monitored and evaluated to mitigate threats, from infrastructural risks to potential large-scale failures.

For this, organizations need a multidisciplinary approach. Legal professionals are essential to analyze applicable laws and determine compliance scope, while machine learning engineers, data scientists, and AI governance professionals play a crucial role in translating these requirements into actionable technical and operational measures. 

This represents a completely new domain of governance, where legal compliance intersects with technical AI implementation, requiring specialized expertise to ensure that AI systems are not only compliant but also responsibly designed and managed from the ground up.

3. Moving Forward: Building Rigorous AI Governance

To address these new and complex risks, a fresh governance approach tailored specifically to AI is essential. It should include:

• New Skills and Roles: Traditional governance teams may not have the specialized skills necessary to understand and manage AI systems. AI governance requires people with expertise in data science, machine learning, ethics, and regulatory compliance. For example, ML engineers, data scientists, and AI governance specialists can provide insight into potential model risks and biases.
• Processes for AI-Specific Risks: Unlike traditional software, AI models continuously evolve. Governance must therefore include processes for regular model reviews, audits, and performance evaluations. MLOps tools can help teams monitor AI models and identify any potential drift in accuracy or bias, ensuring ongoing compliance and reliability.
• Advanced Tools and Technologies: Specialized governance tools are necessary to handle the unique requirements of AI. The Ethical AI Database - the only publicly available, vetted database of AI startups providing responsible AI enablement services - is currently tracking 301 companies across seven categories, from AI inventory and compliance platforms, AI risks management services, to AI data privacy, AI security tools, AI explainability and bias detection tools tailored for AI models, filling in gaps that traditional tools can’t address.

4. Conclusion: Adapting to New Realities in AI Governance

The rapid integration of AI into business operations has brought about risks that are unfamiliar to traditional governance structures. In the face of the rapid AI tool adoption across organizations and the emergence of multi-agentic AI systems it becomes even more clear that legacy governance frameworks are not sufficient. 

The unique risks posed by AI systems are not theoretical; they have significant real-world implications. Poorly governed AI systems can directly impact brand reputation, erode public trust, and result in costly legal repercussions.

Moving forward, companies must prioritize building governance structures that encompass the specialized skills, processes, and tools required to address the distinct and complex risks introduced by AI.

Boards and executives who adopt this forward-looking approach to AI governance can position their organizations not only to avoid costly pitfalls but also to gain a strategic advantage in a rapidly evolving digital landscape. This includes understanding AI risks as a company-wide issue and not a departmental risk. AI risk need to report into enterprise risk management and be included in Business Continuity Planning. Additionally, with AI governance in place, organizations have an excellent foundation for AI product management and strategy, and a partner that can provide an overview of all your organization's AI assets.

Investing in AI governance is about more than compliance; it’s about ensuring that AI serves as a responsible and beneficial asset to the company and its stakeholders.