Hitting an AI ceiling? Watch for these signs

Hi folks,

I hope 2025 has got off to a good start for you. That new year appetite for change is very apparent – already, I've been having a lot of conversations with other support leaders who've successfully introduced AI to their teams, but are now hitting a ceiling. 

They can see the potential for doing more, but their teams are spending too much time tweaking AI responses or managing scattered documentation. They know roles like knowledge managers and conversation designers could help, but they're wrestling with questions about timing and resources.

It's a tricky balance – you want to be proactive about creating these positions, but you also need to justify the investment, whether you're looking to retrain existing team members or bring in new expertise.

After working through this transition with my own team and talking to others who've done the same, I've noticed some clear patterns that signal when it's time to take this step. Here are some key indicators to look for.

Signs it's time for a knowledge manager

Your documentation is more of a headache than a help

• Content creation and updates are manual and tiresome for your team, with little to no AI inspiration to remove the toil.
• Knowledge base articles require frequent updates to keep pace with product changes – and there’s often a time lag with getting these updates made.
• Your content lacks organizational structure and governance – multiple versions of similar information exist across different platforms, which makes everything feel messy and hard to keep track of.

You’re disappointed with how your AI agent is performing

• You believe in the power of an AI agent because you've seen successful results when you’ve tested it with a sample of high-quality content, but you know other content isn't up to scratch and/or has huge gaps. Because of this, your AI agent is consistently misinterpreting or providing outdated information when drawing from your knowledge base.
• Your AI agent struggles to retrieve information from knowledge articles which haven't been properly formatted for AI to parse.
• Metadata and tagging systems are inconsistent or non-existent.

Team productivity is suffering

• Members of your team spend a notable chunk of their time wrestling with documentation and there's no clear strategy in place for longer term management.
• Training new team members takes longer because documentation is confusing and scattered everywhere.
• Multiple team members maintain personal notes or unofficial documentation because it’s easier than dealing with your knowledge base.

Signs it's time for a conversation designer

Response quality is inconsistent

• Your human team members spend significant time clearing up the "mess" of a poorly implemented AI agent or automation flow.
• You don’t have clear frameworks for managing the hand-off between your AI agent and human team, which leaves customers confused about who they’re talking to.
• Your AI agent struggles to stay consistent with brand guidelines and apply the right style and tone of voice to customer communications.
• Depending on the customer segment or issue type, you want to determine bespoke experiences, while still using your AI agent as the first port of call.

Channel complexity is increasing

• Your support volume is increasing and/or you have a growing number of customer interaction channels, all of which require different response styles.
• You’re facing rising complexity in product features that require specialized response handling. 
• Multilingual support needs are creating challenges in maintaining consistent messaging.

Your team's focus is split across too many responsibilities

• You have team members whose time and focus are split between frontline work with customers and managing AI and automation. Or, you have operations teammates who wear too many hats with responsibilities stretched across areas like analytics, enablement and QA, as well as AI and automation.
• Your AI agent CSAT is significantly lower than your human team's CSAT, and you don't have full insights into why or an action plan for how to increase it.

Every team's journey with AI looks a little different, but once you start seeing these patterns, you'll have a clearer picture of your next move. Some teams opt for brand new headcount, others convert some of their frontline roles into these “force multiplier” positions. 

If you're curious about what these roles actually look like in practice, our own Senior Knowledge Manager, Beth-Ann Sher , and Conversation Designer, Fred W. , shared details about what a “day-in-the-life” looks like for them and how they've helped shape Intercom’s internal AI support operations on our customer service podcast, The Ticket. Their insights will give you a better sense of how these roles might fit into your team.

Here’s to knowing when it's time to add more focus to your AI-first strategy – and having the confidence to take that step!

Ruth O'Brien

Senior Director, Automated & Proactive Support

Was this forwarded to you by a friend?

Sign up, and get your own copy of our support leaders' takes on AI and customer service every two weeks!

👉 Subscribe

What we've been up to...

The four pillars of CX: What makes for great customer experiences

According to  CXChronicles founder Adrian Brady-Cesana , excellent customer experiences all come down to how you set up your team, tools, processes, and feedback loops.  

Listen now

Join the Intercom Expert Program

Our brand new Expert Program is live, and we’d love for you to be part of it. You’ll get to connect with other Intercom users, share your expertise, gain recognition, and get your hands on exclusive swag as a thanks for your contributions. 

Find out more

Poll: What’s the most underutilized data source for improving CS?

From chat transcripts to frontline support insights and CSAT scores, there are so many avenues to learn more about the people we serve. What do you think is the most underrated source of info? 

Go to poll

What's happening in customer service?

Understanding and preparing for the 7 levels of AI agents

For CS leaders grappling with AI strategy, this framework from Forbes offers a clear view of how AI agents evolve from simple chatbots to sophisticated systems. The seven-level roadmap helps benchmark your current capabilities and identify practical next steps.

Read now

Raw and unfiltered: 8 Reddit communities where CX professionals keep it real in 2025

Looking for insights from folks on the frontlines? Reddit is the place to go. In this CX Today post, Rob Scott has rounded up eight valuable subreddits where real conversations about customer experience are happening.

Read more

Four digital customer success trends to watch in 2025

If you're working on your digital strategy, this ChurnZero piece outlines four key trends to pay attention to. From focusing on efficacy as well as efficiency, to (of course) deepening investment in AI, you’ll find some valuable insights into where digital CS is heading this year.

Read more

👉 To get the full value of The Ticket, with all your essential Customer Service intel both inside and outside of Intercom, subscribe to the email version of The Ticket here!