Crafting a GenAI Solution Architecture: Essential Components

In this blog, we will explore the process of crafting a GenAI solution architecture and uncover the essential building blocks for success.

Every new technology or framework/product experiences a "Honeymoon phase," characterized by the allure of novelty and excitement, coupled with minimal expectations as users familiarize themselves with the technology and its potential business impact.

In the beginning, we hit the right industry terms and concepts, and talk about fancy solutions that can impact business positively without having to go into details on how you actually can achieve it in the real world, it is a matter of time before business asks, "How is it helping me?"

GenAI has reached a stage where the honeymoon is about to be over; it's time to focus on real-world architectures and solutions that we can implement across customers.

While some of the big organizations have already started to showcase the business value of GenAI and its related tools, others are struggling to realize the same.

My R&D

Based on my last 3 months of learning, ideation, R&D and my curiosity, i have tried to put across a set of key components that you need to have in a solution which you can call a GenAI-driven solution.

Just using Azure OpenAI, ChatGPT, Google Gemini, or other ready-to-use platforms, doesn't make it a GenAI solution.

I am intentionally not putting a block diagram of the architecture, as it deserves a detailed post for itself.

But before anything

Before going into technical architecture, there are two important topics that you need to have clarity on. The Business problem and what specific area of the problem you are trying to solve.

Clarity on Business Problem

The first and only thing you need to have clarity on is what problem you are trying to solve. Has someone already solved this problem, or is it still unresolved? It's crucial to have a clear understanding of the problem and whether it can be addressed without relying on GenAI.

Answers to these questions are key to ensuring we define the right expectations for the outcome.

Focus Area on Business Problem

Once you identify the problem, depending on whether it's already solved with the existing system or if you're attempting to solve it for the first time (which is usually the case), you need to identify the key focus of the solution outcome.

• Cost
• Volume
• Performance
• Accuracy

The technical details that you identify & design, depend on answers to the above questions and focus.

With the business problem identified and focus on the key outcome of the GenAI solution, below are key components that you need to include in your architecture to get to a GenAI architecture instead of platform architecture,

Data Sources

You need to include problem-specific data sources, including the format of the data, the data attributes and different sources where you can retrieve the data and inject it into a GenAI-driven solution.

The data source can be a database, PDF, excel, web content or any other format that the business sees as fit and relevant to use.

Prompt Templates

Mapping of prompts to data to identify prospective prompt templates based on user personas and business domains. The prompts can be specific or templatized with placeholder values for dynamic data.

SLM - LLM Combo

By now, you would have heard "Large Language Models" quite often to get used to it, but rarely hear what are known as SLMs (Small Language Models).

SLMs allow businesses to utilize results which are similar compared to LLMs at optimized cost and performance.

Some of the SLMs such as Phi3 - Mini are so powerful yet, small, that it can run. in your laptops.

A typical architecture should include a combination of SLMs and LLMs for an optimized approach to cost and performance. Cost and Performance optimization is key for any architecture.

Chaining

A GenAI solution cannot in most cases be achieved with single SLMs or LLM. It will be a combination of different LMs working together with specific input and output to reach the specific results. This is known as prompt chaining and other names. Identify the chaining approach and sequence, sometimes making the solution delivery intended results at the most optimized cost.

Platform to bring all these together.

Last, but not least, you need to identify a platform to connect all together to ensure development, monitoring and going live at ease.

Here is where platforms such as OpenAI, Azure OpenAI, Gemini, and Lang chain come into the picture. These platforms make solution development faster and easier with ready-to-use models.

Conclusion

As I said, at the beginning, a GenAI solution is not about using the most popular platform. It is about going back to basics and designing a solution that utilizes various building blocks of the technology and customizes the same for your business needs. This approach is what best stands a chance of providing the value that we all trying to achieve for our customers.

Feel free to correct me or add anything I missed. Happy to share and learn.