More Related Content
Similar to AI LLM Inference and SageMaker Pipeline in AWS (20)
Recently uploaded (20)
AI LLM Inference and SageMaker Pipeline in AWS
- 1. AI LLM Inference on AWS EMRE GÜNDOĞDU
- 2. About • Graduated Computer Engineering on Kocaeli University Education • My Previous experiences are open source systems management on different companies over 10 year Experience • Working on Unit of CCS at the InsightHub project as Chief Cloud Engineer, responsible of AWS, Azure and Local Private Cloud management Current
- 3. Ingredients AI and ML Service on AWS Artificial Intelligence Process Demo for LLM Microservice Deployment on AWS Demo Entire MLOps Pipeline Deployment on AWS
- 4. AWS SERVICE FOR AI AND ML PROCESS ArtificaI Intelligence Services • App Developers and No ML Required Machine Learning Services • ML Developers and Data Scientists ML Frameworks & Infrastructure • ML Researchers and Academics
- 5. AWS AI Stack AI Services Amazon Comprehend Amazon Lex Amazon Polly Amazon Rekognition Amazon Translate Amazon Transcribe Amazon Personalize ML Services Amazon SageMaker Ground Truth Notebooks Training Hosting Algorithms Marketplace ML Frameworks & Infrastructure Frameworks Interfaces AWS GreenGrass AWS EC2 AWS DeepLearning AMI
- 6. AI Services • WHAT • Natural Language Processing (NLP) service that finds insight and relationships within text • WHEN • Sentiment analysis of social media post Amazon Comprehend • WHAT • Combines time-series data with other variables to deliver highly accurate forecasts • WHEN • Forecast seasonal demand for a specific color of shirt Amazon Forecast
- 7. AI Services • WHAT • Build conversational interfaces that can understand the intent and context of natural speech • WHEN • Create a customer service chatbot to automatically handle routine requests Amazon Lex • WHAT • Recommendation engine as a service based on demographic and behavioral data • WHEN • Provide potential upsell products at checkout during a web transaction Amazon Personalize
- 8. AI Services • WHAT • Text-to-Speech service supporting multiple languages, accents and voices • WHEN • Provide dynamically generated personalized voice response for inbound callers Amazon Polly • WHAT • Image and video analysis to parse and recognize objects, people, activities and facial expressions • WHEN • Provide an additional form of employee authentication though facial recognition as they scan an access badge Amazon Rekognition
- 9. AI Services • WHAT • Extract text, context and metadata from scanned document • WHEN • Automatically digitize and process physical paper forms Amazon Textract • WHAT • Speech-to-Text as a service • WHEN • Automatically create transcripts of recorded presentation Amazon Transcribe • WHAT • Translate text to and from many different languages • WHEN • Dynamically create localized web content for users based on their geography Amazon Translate
- 12. Machine Learning Services •An integrated development environment (IDE) for data science and machine learning. It allows users to build, train, and deploy ML models using Jupyter notebooks in a collaborative interface SageMaker Studio •Managed service for training machine learning models at scale. It allows you to train models on a variety of instance types and frameworks while handling infrastructure management. SageMaker Training •Used for running data processing jobs, such as data pre-processing, post- processing, feature engineering, and model evaluation, using custom scripts. SageMaker Processing •Managed deployment for machine learning models in a production environment. It provides several types of inference endpoints SageMaker Inference •A central repository for cataloging models in different stages (e.g., staging, production). It allows you to track model versions and metadata, and simplifies model deployment workflows. SageMaker Model Registry •Automatically builds and trains the best machine learning models from your dataset. It automates data pre-processing, model selection, tuning, and evaluation. SageMaker Autopilot •A data preparation tool that simplifies the process of cleaning, transforming, and visualizing data without writing code. It integrates with SageMaker Studio for seamless workflows. SageMaker Data Wrangler •Tracks and organizes machine learning experiments. It captures parameters, metrics, and metadata from training jobs to help manage and compare different models. SageMaker Experiments •A data labeling service that helps create highly accurate labeled datasets by combining machine learning with human input. It offers built-in workflows for common labeling tasks SageMaker Ground Truth •A fully managed service for building, automating, and managing end-to-end machine learning workflows. It enables you to create reusable pipelines for data preparation, model training, and deployment SageMaker Pipelines •A managed repository for storing, managing, and retrieving machine learning features. It supports real- time and batch processing, and helps with feature reuse across different models SageMaker Feature Store •Provides a library of pre-built machine learning models and solutions. It enables you to quickly start using popular models for tasks like text classification, image recognition, and more SageMaker JumpStart
- 13. AI PROCESS The typical ML pipeline is comprised of: Data Preparation (collection, cleaning/pre-processing, feature engineering) Model Training (model selection, architecture, hyperparameter tuning) CI/CD, Model Registry (storage) Model Serving Observability (usage load, model drift, security)
- 14. DEMO DEMO LLM Deployment on AWS Microservice
- 15. ML Workflow – SageMaker Pipeline Challenges with MLOps • Project management • Communication and collaboration • Everything is code • CI/CD • Monitoring and logging Benefits of MLOps • Productivity • Repeatability • Reliability • Auditability • Data and model quality
- 16. MLFlow Pipeline
- 17. Example Pipeline Preprocessing Training Evaluation (Processing) Condition RegisterMode) CreateModel
- 18. SageMaker Project with GitLab
- 19. DEMO Demo Entire MLOps Pipeline Deployment on AWS