Srushti Rashmi Shirish

❝ How do you taste your Wine? With the right Glass, served from the right Bottle. How do you utilize your LLM? With the right Task, served from the right Compute environment. ❞ If the quote above caught your interest, check out my blog post that was featured on the Toyota Connected India Medium channel last week: Leveraging LLMs in Cloud - 🔗 https://lnkd.in/gfHsdevH --- I hope this blog adds a bit of amusement for the seasoned AI/ML engineers 🧠👨‍💻 with the carefully crafted images and perhaps even the silly wine analogies 😉. Whether you're a cloud engineer ☁️ 👨‍💻, a product owner of cloud-based apps, or someone new to Generative AI in the cloud, this blog offers a detailed, beginner-friendly look into the selection and deployment of LLMs in cloud environments. --- A heartfelt thanks 🙏 to my colleagues at Toyota Connected India for taking the time to review, shape this piece and even share a creative writeup like below in the original post 🫡 👇 🤔 Curious to dive in? Click the link below to explore the blog on Medium! 🌐 🔗 https://lnkd.in/gfHsdevH

18

🌟 **Decoding Vector vs. Graph Databases for RAG & RAFT – Which One Wins?** When it comes to enhancing large language models through Retrieval Augmented Generation (RAG) or Retrieval Augmented Fine-tuning (RAFT), the choice of database—vector or graph—plays a pivotal role. 🔹**Vector Databases:** Data is partitioned into chunks and each chunk is transformed into a vector using a language model. These vectors are then indexed in a vector database, facilitating quick nearest neighbor searches to find the most relevant information chunks in response to a query. 🔸**Graph Database:** This database shines in storing relationships between entities. By parsing and indexing these connections, graph databases excel at constructing detailed subgraphs for specific queries, ensuring that only the most relevant, contextually connected information is retrieved. It’s about understanding the network of relationships.An LLM is good at extracting the below kind of triplet information. [Entity A] -> [Relationship/Context] -> [Entity B] Both have their strengths—vector databases are great for fuzzy matching large data sets quickly, while graph databases excel in precise context and relationship extraction. 🚀 **The cutting edge? Combining both!** : This hybrid model takes advantage of the quick searching capability of vector databases with the deep relational insights of graph databases. For RAG or RAFT, this means not only retrieving the most relevant information quickly but also understanding its context within a broader knowledge base. For example, if a model is answering a query about "healthy eating," a vector database can quickly fetch documents mentioning health and nutrition. A graph database then elaborates on how these concepts connect with diet plans, medical advice, and recipes, providing a comprehensive, context-rich response. #GraphRAG #RAG #RAFT #GenerativeAI #GraphDB #VectorDB

13

Top 3 Learnings from Chip Huyen O'reilly podcast. 🌟 2 reasons limiting most of the companies from using Gen AI in Customer facing applications: Reproducibility, Compliance (GDPR,EU AI ACT etc) 🌟 2 ways most companies approach Gen AI: Problem first: We have this problem , Can we use Gen AI to solve it better . Chaos Monkey Research: Ask all your Senior to start using Gen AI foundation models, test and come up with new use cases But having Applied Research team for LLM's doesn't make it sense 🌟AI Engineering : Because we have good foundation models and spend lot less time on model development AI engineers are expected to be full stack engineers (end to end) with knowledge of front end engineering etc https://lnkd.in/ez7TEQAC

4

How are Graph Convolutional Neural Networks (GCNNs) a generalisation of Convolutional Neural Networks (CNNs)? My next blog in the Graph series on "Deep Learning with Graphs" is on the way (well, it's already written on paper) and it's going to be a blockbuster. The beauty of Graph Machine Learning is that the more you read about it, the more you feel that your neurons (real ones!) are learning to think better! And you're already in love with it! Coming to the point: One might recall the convolution operations in a CNN. You have a grid of pixels representing an image. You have a 3 x 3 filter. You slide the filter over the image and carry out aggregation to obtain a new pixel value. You proceed in the same fashion over the whole grid, sliding towards the right , bottom, carrying out the convolution operation and finally getting a new transformed, reduced image through the series of aggregations. You repeat this through the different convolutional layers which together form a Convolutional Neural Network Now think of a Graph. It is not simple grid like an image. The topology of a Graph is arbitrary. There is no notion of left, bottom, right in a Graph like in an image which is represented through a grid of pixels. You cannot slide over a Graph as simplistically you did and performed the convolutional operations in an image. How do you approach the problem? How do you carry out convolutional operations in a Graph? Now take a node in the Graph. Take all the nodes which are the immediate neighbours of this node. The convolutional operation here is: You aggregate the information from all the neighbours and pass it to the node. The information is nothing but the emedfings of the neighbour nodes. You add together the emedding of the node under consideration. You transform the aggregated embedding by multiplying with a weight Matrix and transform the node's own embedding by multiplying with a different weight matrix. This is the typical linear transformation operation in a NN. You apply non linearity - the activation function like ReLu. Now think about a CNN taking the same grid and considering a central pixel. Consider the rest of the pixels around the central pixel. You are essentially doing the same thing: taking information from the neighbour pixels along with the central pixel, aggregating, as I described above for the node in the Graph. Hence a Graph Convolutional Network is a generalisation of CNN The story goes, you do the same for every node on the Graph. Each node in the Graph represented through a different neural network architecture! This is amazing 🔥🔥🔥💓💓💓 #graphs #machinelearning #deeplearning My Technical Blogs Library: 1) https://lnkd.in/ejyWacXn 2) https://lnkd.in/bfrkRjq Follow Ajay Taneja for insights on: Data Science, Data Engineering, Finite Element Analysis, Non Linear Finite Element Analysis, Fatigue, Fracture Mechanics

22

Elevate Your Q&A with LangChain's RAG LangChain's latest guide explores how to build sophisticated Q&A applications using Retrieval Augmented Generation (RAG). This method enhances LLMs by integrating private or post-cutoff data, making them more effective for specific queries. The guide covers the RAG architecture, including indexing data and the retrieval-generation process, ensuring comprehensive understanding for developers. Key topics include: · RAG fundamentals and architecture · Data indexing and retrieval · Enhancing LLMs with specific information · RAG Architecture Breakdown The RAG architecture combines retrieval and generation capabilities to improve the accuracy and relevance of answers provided by language models. The architecture involves two main components: · Retriever: This component is responsible for searching a large corpus of documents to find relevant information based on the input query. It indexes the data and retrieves documents that are most likely to contain the answer. · Generator: Once relevant documents are retrieved, the generator uses these documents to generate a precise and coherent answer. This integration ensures that the answers are not only contextually accurate but also grounded in the specific data available. By combining these components, RAG leverages both the extensive knowledge embedded in LLMs and the specific information contained in private or domain-specific datasets. This hybrid approach ensures that the generated responses are both relevant and up-to-date, making RAG a powerful tool for developing advanced Q&A systems. https://lnkd.in/df5uAE7K #AI #MachineLearning #LangChain #DataScience #QandA

3

1件のコメント

𝗟𝗶𝗴𝗵𝘁 𝗻𝗼𝘁𝗲𝘀 𝗳𝗼𝗿 𝗮 𝗠𝗼𝗻𝗱𝗮𝘆 𝗲𝘃𝗲𝗻𝗶𝗻𝗴: I had been reading through Advanced RAG techniques and GraphRAGs late evening yesterday in perparation of my next blogs on LinkedIn. Some points that caught my attention: In order to improve the retrieval accuracy in the Retrieval-Augmented Generation (RAG) approach, we may use: ⭕Re-Ranking of Context: we use LLMs like GPT-4 or specialised models like ColBERT or Cohere to re-rank the context and then pass only the top most relevant contetxts to the LLM to get the final answer. 𝗧𝗵𝘂𝘀, 𝘄𝗲 𝘂𝘀𝗲 𝗮𝗻 𝗟𝗟𝗠 𝘁𝗼 𝗿𝗲-𝗿𝗮𝗻𝗸. 𝗧𝗵𝗲 𝗶𝗱𝗲𝗮 𝗯𝗲𝗶𝗻𝗴 𝘁𝗼 𝗶𝗺𝗽𝗿𝗼𝘃𝗲 𝘁𝗵𝗲 𝗰𝗼𝗻𝘁𝗲𝘅𝘁 𝗽𝗮𝘀𝘀𝗲𝗱 𝘁𝗼 𝘁𝗵𝗲 𝗟𝗟𝗠 ⭕GraphRAGS: Going from Community Summaries -> Community Answers ->Global Answers 📌 Fundamentally in a GraphRAG , we get chunks out of the source documents as in the standard RAG process 📌 We then carry out Entity Extraction and Relationship Capture from these chunks. This is essentially building up of Nodes and Edges of the Knowledge Graph. 📌 We carry out Element Summmaries: This is the summarization of all the nodes and relationships extracted. 📌 The community summaries are randomnly shuffled and the reason for shuffling being that they shouldn't be concentrated in a single context window. 📌 We make multiple calls to the LLM to answer the query against each of the chunk of the community summaries. 𝗔𝗻𝗱 𝗳𝗼𝗿 𝗲𝗮𝗰𝗵 𝗰𝗵𝘂𝗻𝗸 𝘄𝗲 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗲 𝗶𝗻𝘁𝗲𝗿𝗺𝗲𝗱𝗶𝗮𝘁𝗲 𝗮𝗻𝘀𝘄𝗲𝗿𝘀 𝗮𝗻𝗱 𝗲𝗮𝗰𝗵 𝗮𝗻𝘀𝘄𝗲𝗿 𝗶𝘀 𝘀𝗰𝗼𝗿𝗲𝗱 𝗮𝗻𝗱 𝗿𝗮𝗻𝗸𝗲𝗱. 📌𝗔𝗴𝗮𝗶𝗻 𝗮𝗻 𝗟𝗟𝗠 𝗶𝘀 𝘂𝘀𝗲𝗱 𝘁𝗼 𝘀𝗰𝗼𝗿𝗲 𝘁𝗵𝗲 𝗮𝗻𝘀𝘄𝗲𝗿 𝗯𝗲𝘁𝘄𝗲𝗲𝗻 𝟬 𝗮𝗻𝗱 𝟭𝟬𝟬 𝗮𝗻𝗱 𝗳𝗶𝗻𝗮𝗹𝗹𝘆 𝘁𝗵𝗲 𝘁𝗼𝗽 𝗿𝗮𝗻𝗸𝗲𝗱 𝗮𝗻𝘀𝘄𝗲𝗿𝘀 𝗮𝗿𝗲 𝘁𝗮𝗸𝗲𝗻 𝗮𝗻𝗱 𝘂𝘀𝗲𝗱 𝘁𝗼 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗲 𝗮 𝗳𝗶𝗻𝗮𝗹 𝗴𝗹𝗼𝗯𝗮𝗹 𝗮𝗻𝘀𝘄𝗲𝗿. The point I'm trying to make is: The above remined me of ChatGPT training process [https://lnkd.in/dWpuyCpV] . Here, we had human labellers rank the ChatGPT responses based on “Likert Scale” – the ranking being based on the quality of the responses. This was part of the Step 2 of ChatGPT training process. 𝗔𝗻𝗱 𝘁𝗵𝗲 𝘀𝘁𝗼𝗿𝘆 𝗮𝗳𝘁𝗲𝗿 < 𝟮 𝘆𝗲𝗮𝗿𝘀: the human ranking is already replaced by LLM ranking! Coming Soon: ⭕RAG Beyond Basics: Part 15 of my Graph Series of Blogs ⭕The Marriage of Retrieval Augmented Generation with Knowledge Graphs: Part 16 of my Graph Series of Blogs Watch this space! #graphs #deeplearning #rags #llms My Tech Blogs Library 1) https://lnkd.in/ejyWacXn 2) https://lnkd.in/bfrkRjq Follow Ajay Taneja for insights on Data Engg, Data Science, Finite Element Analysis, Non Linear Finite Element Analysis, Fatigue, Fracture Mechanics Image credits: Claudia Sonnet 3.5!

12

Join me and Farris in Neurips workshop on Fine-Tuning in Modern Machine Learning: Principles and Scalability. Our paper title is: Optimizing Small Language Models for In-Vehicle Function-Calling

25

#Llama3 is released now. Some insights: 1. 15T tokens processed 2. 8B and 70B instruct and fine-tuned variants 3. 400B variant under training (multimodal) 4. 24,576 NVIDIA H100 Tensor Core GPUs linked with an NVIDIA Quantum-2 InfiniBand network. model-card: https://lnkd.in/dJzjzdmW #AGI #Llama3 #GenAI #opensource

4

very informative piece on Iterative DPO, CoT and RPO by @Philipp Schimid

1

#LLMOpsSeries #Day9 🔍 LLMOps Unveiled: Feature Engineering from Text Data 🔍 In today's LLMOps post, we’ll explore various methods for feature engineering from text data. These techniques are essential for extracting meaningful features that enhance the performance of large language models. TF-IDF (Term Frequency-Inverse Document Frequency): Measuring the importance of words in a document relative to a corpus. This helps in identifying key terms that are significant within the text. Word Embeddings: Representing words as dense vectors using techniques like Word2Vec, GloVe, or FastText. These embeddings capture semantic relationships between words. N-grams: Extracting contiguous sequences of n words to capture context. This technique helps in understanding the sequence and co-occurrence of words. POS Tagging (Part of Speech Tagging): Identifying parts of speech to enrich text data with syntactic information. This aids in understanding the grammatical structure of the text. Named Entity Recognition (NER): Identifying and categorizing entities such as names, dates, and locations. This helps in extracting and utilizing specific information from the text. By applying these feature engineering techniques, you can enhance the input data’s expressiveness, leading to better performance and insights from your large language models. Stay tuned for our next post, where we’ll discuss the main challenges when developing LLMs for production. #LLMOps #AI #MachineLearning #DataScience #ArtificialIntelligence

1

🎤🎤𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 𝗔𝘂𝗴𝗺𝗲𝗻𝘁𝗲𝗱 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻 𝘄𝗶𝘁𝗵 𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 𝗚𝗿𝗮𝗽𝗵𝘀 What's next in my Graph Series of blogs after Graph Convolutional Neural Networks? I'm happy to announce that following the release of my next blog on "Deep Learning with Graphs" in my ongoing Graph Series, I will next be publishing an elaborate article on: Retrieval Augmented Generation (RAG) Using Knowledge Graphs This is a very important and fast progressing concept and will go a long way. RAG as such works by converting the document into vectors and then doing a similarity search between the query and the document embeddings and passing the context to an LLM to retrieve the appropriate answers. However, Graphs will give much better results for searches involving several hops: one may recall the concept involving Breadth First Search and Depth First Search in Graphs that I discussed in my blog on Node2Vec in part 5/6/7 of the series. The context of a node in a Graph is derived from its immediate neighbours as well as the nodes that may lie a few hops away and it is exactly here that RAG with Knowledge Graphs scores over the conventional approach. What happens in RAG with Knowledge Graphs? In RAG with Knowledge Graphs we first construct a Knowledge Graphs by passing the unstructured or semi structured text data to an LLM and ask the LLM to generate the entities and relationships between those entities from the chain of documents. Once these entities and relationships are extracted these are use to construct a Knowledge Graph. Then, following the construction of the Knowledge Graph, the textual prompt of the user can be converted into a Graph Query Language and used to query the Knowledge Graph The approach scores over the conventional RAG approach because the search involves several hops of a node and the response will be much more context specific than using RAG ina conventional form I'm planning to have this blog with practical examples but of course the theory is super important . Watch this space! #graphs #llms #RAG #integration My Technical Blogs Library: 1) https://lnkd.in/ejyWacXn 2) https://lnkd.in/bfrkRjq Follow Ajay Taneja for insights on: Data Science, Data Engineering, Finite Element Analysis, Non Linear Finite Element Analysis, Fatigue, Fracture Mechanics

7

Myths about Large Language Models (LLMs) like #Chatgpt #Gemini #llama and others - LLMs can understand like humans: LLMs work on pattern matching using algorithms. LLMs cannot understand the questions as humans do and so the response might not be always 100% as expected. - Accuracy of LLMs:While models try for accuracy, the information given might be incorrect or misleading. Model does not verify facts of the response - More data will improve response of LLMs:While more data can improve quality to some extent, fine tuning, transfer learning, tunning some parameters, modifying architecture of data flow, etc. can improve performance of LLMs. - LLMs can be creative like humans: LLMs can generate text, images or videos. While LLMs can augment human creativity but it cannot replace complexity of human creativity. - LLMs are invulnerable to errors or manipulation: LLMs can make mistakes and can be manipulated with adverse inputs to generate misleading information. - LLMs will surpass human intelligence: Artificial general intelligence (AGI) is still far off from reality and hence LLMs are far from achieving human-like intelligence. * views expressed are personal

42

3件のコメント

In my early learning phase, I often confused Layer Normalization and Batch Normalization—I knew they improved numerical stability, but the key differences eluded me. Watching Umar Jamil's videos clarified everything: Batch Normalization works across the batch dimension, ideal for CNNs, while Layer Normalization focuses on features, perfect for RNNs and Transformers. Grateful for such insightful resources! 🙌 #deeplearning #machineLearning #GENAI #Transformer

10

🌿 SandalQuest: Digitizing Knowledge of Sandalwood Cultivation with AI 🌿 I’m excited to share our project SandalQuest, created as part of the ML-Fiesta: AI/ML Hackathon organized by International Institute of Information Technology Bangalore! Our team developed an AI-powered pipeline to preserve and provide easy access to the indigenous knowledge surrounding sandalwood cultivation in Karnataka. 🔑 What We Built: - An Automatic Speech Recognition (ASR) model tailored for colloquial Kannada, capturing informal language and dialect variations. - A speech-based question-answering system that allows users to ask questions and retrieve relevant answers from a transcribed audio database. - A searchable audio database that helps users access traditional knowledge through speech. 💻 Tech Stack: - Python, PyTorch for AI & Machine Learning - Whisper for Speech Recognition - Hugging Face Transformers for NLP - MongoDB for database management 🔍 Impact: We’re helping preserve valuable knowledge on sandalwood cultivation and making it easily accessible for farmers, researchers, and enthusiasts. 🔗 Check out the project: https://lnkd.in/dMxRP76s 🚀 Team Name: Code Wizards Team Members: - Shani Sinojiya (AI/ML & Backend Developer) - Mohammad Anas Africawala (AI/ML Engineer) - Tisha Patel (Full Stack Developer) It’s been an amazing experience working on this project, and I’m proud of what we’ve achieved. Looking forward to applying these insights to real-world applications! #AI #MachineLearning #SpeechRecognition #Kannada #ColloquialSpeech #Sandalwood #DataScience #NaturalLanguageProcessing #AIForGood #Hackathon #TechForGood #GoogleCloud #AIResearch #DeepLearning

10

2件のコメント

Excited to share our interactive BlogApp on Generalized Linear Models (GLMs)! In collaboration with Venkatachalam Subramanian Periya Subbu, Satya Lasya N, and Pooja Baralu Umesh, we've developed an intuitive tool that makes understanding GLMs both engaging and accessible. If you are an aspiring Data Scientist, Actuary, Statistician, Economist, Epidemiologists or an Engineer, you definitely need to check it out at: https://lnkd.in/gMqf3M5Y View the source code: https://lnkd.in/g9HUm_7r Special thanks to Professor Cody Carroll for the guidance throughout this project. Let's discuss GLMs! What's your experience with these powerful statistical tools? Share your thoughts below. #DataScience #Statistics #MachineLearning #GLM #Analytics #Programming #USF #MSDS

34

2件のコメント