The Team Is Offering an "AI Product". Let’s See What You're Getting Into!
Chaim Bechor
Owner @ Self-employed | Business Automation Expert using AI and Low Code Tools
Intro
As businesses hurry to join the "AI Revolution,” managers face the challenge of deciding whether a new product or a new version of an established product is within their company's scope. Years of experience in software projects suddenly seem irrelevant, and the new turf has more unknowns than knowns. How much will it cost? How long will it take? Is this going to be a profitable, competition-ready product?
The media's extreme terms regarding “AI” confuse and sometimes even instill fear: Marketing urges the addition of "AI" to the company’s offerings, claiming the window of opportunity is closing quickly, while R&D floods you with technologies and ideas.
The following post presents a model that clarifies the challenges and elements involved. It is based on allocating project types into a hierarchical structure, in which each layer represents a different level of complexity and carries a varying level of effort and risk. It is meant for non-technical decision-makers and is based on my experience with such projects, analyzing them as a product manager and implementing quite a few.
Proposed Model
The model I use to analyze AI-based projects is illustrated below, using a triangle form to represent the relative number of ongoing projects (width) and increasing level of complexity - top (max) to bottom:
Innovating at the AI model level
The leading players in this section—the “big guys”—OpenAI, MicroSoft, Google, and Meta (Recently Joined by the Chinese - DeepSeek and Alibaba) invest billions in new levels of training and in an attempt to improve the principles upon which these models are built, such as adding “reasoning” ability and various parameter utilization models, designed to improve efficiency.
Silicon-targeted companies, such as Groq (not to be confused with xAi’s Grok…!) offer an LLM-optimized chip that accelerates response and reduces cost and delay by a factor of 10 and more.
Smaller groups and academic projects are trying to innovate by developing smaller models tailored to specific tasks.
Opportunities at the Model Level
While competing with the big guys seems (and is) pointless, some players bet on alternative ways to gain an edge:
Using proprietary data to extend or improve the training set: This specificity offers a different model better suited for a specific domain. Adobe, for example, allows users to create generative AI images based solely on licensed material, which seems vital for the pro community, its primary target market.
Using an innovative approach to reasoning: Reasoning, recently added to all major LLMs, is new and yet to be explored. Unlike the basic mechanism of LLMs, such as transformers, which rely heavily on advanced mathematics, reasoning is based on various forms of logic and, therefore, open to a much wider range of alternatives.
Competing at the Model level is risky, resource-intensive, and therefore suitable for a small group of players. Most AI-related activity is currently focused on the other two layers.
Innovating at the Platform level
As soon as the software industry realized the potential of Generative AI, it was clear that it needed to be integrated into traditional software solutions to exploit its full potential. Access to data, UIs, and links to existing systems all require additional software to be applied while using the LLM core to achieve data processing functions that are impossible to implement with traditional software. It was clear that there is a need for new platforms as the traditional ones were not optimized for the purpose.
What is an AI Apps Platform?
The software industry has been using platforms and frameworks for many years as a proven method to shorten development time and increase the quality of the outcome. From ubiquitous names like React.js and Spring to less-known ones, hundreds of such bundles have been applied to gain speed and convenience.
Generative AI has not escaped this trend, and over the last 24 months, numerous such frameworks and platforms have been introduced, sometimes on a daily basis.
RAGs
The first and real need requiring a new architecture was to extend the LLM's knowledge base, which was limited to its training data with proprietary data. This is achieved now using the RAG architecture. RAG, which stands for Retrieval-Augmented Generation, enhances the capabilities of the language model by integrating auxiliary “on the fly” retrieval mechanisms, improving accuracy and bringing flexibility to the otherwise highly general and unpredictable LLM’s Generative response.
The “Agentic” Era
Prompting—chat input to guide LLMs towards needed answers has become increasingly complex as users discover that specific non-trivial techniques produce better results. Thus, the concept of Prompt Engineering was born. However, prompt engineering, which has introduced terms like "chain of thought" and sophisticated context descriptions, still struggles to overcome the limitations of interactive chat.
This has led to the idea of “autonomous” operation of Generative AI, which creates value by performing actions like traditional software, running in the background. This evolution has given rise to Agents, a combination of interactions between LLMs and additional tools to achieve a predefined goal, all via a single input.
Programming agents required new frameworks or platforms, focusing on LLM interactions rather than UI and database manipulation.
Agentic Platforms
Traditional APIs can easily integrate AI engines with business applications. Agentic frameworks and paforms go beyond these APIs and deal with complex combinations of inter-LLM data communications. Here are notable examples:
LangChain is considered the first widespread Agentic framework. It is an open-source project that provides a convenient and robust framework for complex AI and business app integrations, featuring high performance and scalability.
LangChain has introduced a series of add-ons that provide alternative modes of development and enhanced functionality. Notably, they offer LanGraph, which enables the visual design of the desired functionality.
Visual Development of Agentic Systems
The basic Agentic platforms supplied a wide range of building blocks necessary to code sophisticated Agents. Yet, the need to craft various types of agents called for a simplified environment, and that has been supplied by applying the principles of visual programming rather than coding the entire agent. A notable example is BuildShip, a visual process flow development environment connecting AI APIs to various tools via a drag-and-drop interface.
Buildship and Langchain are now joined by numerous other similar tools, as illustrated in the domain maps described in the site “AI Agents Directory”:
Recommended by LinkedIn
Existing Tools Competing with Dedicated Platforms
While dedicated platforms offer unique adaptations to the characteristics of LLM processing, existing platforms, especially those in the automation field, have joined the rage and started offering solutions of their own. A notable example is an advanced automation platform popular among programmers called n8n. It now has an “Agent” module as part of its action nodes, which can be triggered by various sources and produce agentic output.
Innovating at the Applications Layer
The initial model presented by applications adopting Generative AI was simply imitating the ChatGPT’s chat interface and creating variations in an attempt to add value by emphasizing specific aspects. These “ChatGPT wrappers,” as they are called now, have gained much popularity by simplifying “AI” for many new to the concept. Minor variations such as UI using familiar jargon, pre-prepared prompts, and so on were enough to justify the title “AI application.”
We are not there anymore. Users learned to use the LLMs interface directly, which became more sophisticated over time (e.g., Artifacts in Claude, Projects in Perplexity, and more).
Users now expect AI functionality to be woven into their dedicated apps, offering capabilities that could not be achieved with traditional coding - unstructured text understanding, intention detection, and rich media processing abilities such as text-to-speech, image understanding, etc.
Highly popular groups offered already by multiple plays:
Dedicated “AI Writers”
The first wave of apps introduced following the emergence of ChatGPT managed to create a more focused work environment built for creative text processing. The structured interface and pre-defined set of targeted prompts gained immediate popularity and made their creators a fortune. Some of the more familiar names still leading this section are Jasper AI, Ryter, and Copi.ai.
AI Assistants focused on Productivity:
The ability of Generative AI to handle quite a few tasks associated with productivity, such as automated data management, intelligent, quick answering on a range of queries, and complementing the wiring and calculation abilities of the app user, quickly became inherent in all leading productivity apps, including AirTable, Notion, and all of Google and MS-365 Apps office apps.
Coding-oriented add-ons:
The ability of Generative AI engines to create code and formulas based on free text queries has made them a hit among programmers and data engineers. Notable samples:
Research Assistants
Research is considered one of the best applications of the “Agentic” principle. Agents handling a series of tasks, from planning and searching content to extraction and summary, create a highly usable output in a fraction of the time it would take to perform manually. The proliferation of such solutions is demonstrated in the attached domain map, which only gets bigger every day.
Browser Extensions:
Developers taking advantage of extensions to “read” the browsed page introduced “browsing assistants” allowing summary, query and other forms of page processing to be performed on the spot, in-place browsing, eliminating the need for copying and pasting into a dedicated AI Chat interface such as ChatGPT. The Chrome extensions market offers many implementations, some of which have developed into a complete automation system (i.e. HARPA.ai)
AI-enhanced Chatbots:
Generative AI allowed for converting the classical “tree-based” navigation and answers dominating such products to a sophisticated AI-based bot that can answer free form questions, refer to proprietary documents (through RAG) and exploit rich media forms such as voice.
Business Analysis Enhanced with AI:
As custom-trained AI engines are capable of interpreting and analyzing complex business data through on-the-fly code generation, theri application to BI allows overcoming the limitations of keyword search and classical statistics and provides much more usable outcome.
The rapid adoption of AI is evident in the multitude of applications it enhances. New and improved AI-powered applications are introduced daily, and users embrace them ever-increasingly.
Summary
Introducing AI-based products is still an important business strategy. However, the level of competition and the chosen business strategy must align with the organization's skill set and overall business objectives, rather than simply following trends.
AI-based products will become increasingly common, and AI will often be taken for granted. Achieving a competitive edge will require a carefully crafted strategy that aligns with the organization's strengths and goals.