AI <Connect> Newsletter | Edition #10
Welcome to the 10th edition of the Prescience AI Connect Newsletter. Every week, AI Connect brings you the latest news, trends, insights, experts' opinions, and real-life stories from the AI and data analytics world.
This week in AI Connect, we dive into AWS's latest upgrades to the Bedrock platform, including prompt routing and caching that slash costs by up to 90% and reduce latency by 85%. The update also brings advanced RAG evaluation and seamless LLM integration, boosting its AI capabilities. Meanwhile, Google’s video generator is now available to a wider audience, and Meta has unveiled SPDL, a framework-agnostic data loader optimized for efficient AI model training.
We also share insights from McKinsey on the growing adoption of generative AI across various sectors.
Don’t miss our latest episode of the Prescience Podcast, where we explore the evolving landscape of generative AI, with a focus on proof of concept and prototyping for chatbots and automation tools.
Latest AI Current
Amazon Bedrock
AWS announced caching and intelligent prompt routing for its Bedrock LLM service at re:Invent. Caching reduces redundant processing, cutting costs by up to 90% and latency by 85%, as seen with Adobe’s 72% faster response times. Intelligent routing directs queries to the most suitable model, balancing performance and cost. With growing context windows, these features make large-scale AI deployments more affordable and efficient.
Additionally, Amazon Bedrock introduces two generative AI evaluation tools: RAG Evaluation for app optimization and LLM-as-a-Judge for cost-effective model assessments. Users can create, compare, and review detailed metrics via the console. Now in preview across regions at standard rates.
Google Expands Access to Its Video Generation Tool
Google’s AI video generator, Veo, is now available in private preview for Google Cloud customers using Vertex AI. Veo, which creates 6-second 1080p clips from prompts, will help Quora enhance its Poe chatbot and Mondelez International produce marketing content. Launched in April, Veo supports cinematic styles and editing.
****
Meta introduced SPDL, an AI training model with Thread-Based Data Loading
SPDL is a new framework-agnostic data loader designed for efficient AI model training. Using multi-threading, it delivers 2–3x higher throughput than traditional process-based methods while consuming fewer compute resources, even in standard Python interpreters without free-threading.SPDL also supports Free-Threaded Python, achieving 30% higher throughput with the GIL disabled compared to enabled. This makes it ideal for scaling AI models, ensuring GPUs remain fully utilized as their speed increases.
****
Trends & Insights
According to a report by Mckinsey, organizations are employing gen AI in multiple business functions, with an average of two functions per organization. The most common areas for Gen AI applications are marketing and sales, as well as product and service development.
Recommended by LinkedIn
Quote of the week
“AI systems will evolve into increasingly senior coworkers, gradually taking on longer and more complex tasks as their autonomy grows."
— Sam Altman
Prescience Tidbits
This week's AI Connect episode explores the evolution and deployment of generative AI, focusing on the current phase where companies are prototyping applications like chatbots and process automation tools.
Looking ahead to the expected $15 trillion contribution to the global economy by 2030, driven by advancements in generative AI. The discussion stresses the need for guiding principles and evaluation frameworks to ensure reliability as these technologies scale.
Click here to listen to the full podcast.
About Prescience
Prescience is an AI and data analytics company that works with large Fortune 500 companies and midsized businesses globally, helping them unlock the power of enterprise data, thus enabling faster and better business decision-making.
Founded in 2017, the company offers comprehensive data solutions and services that integrate AI and machine learning across analytics, business intelligence, data engineering, and more—driving measurable business value and ROI for its customers.
You can read some of the customer success stories here.
We want to hear from you about the AI Connect Newsletter. Please share your thoughts, feedback, and ideas in the comments section.
Co-Founder of Altrosyn and DIrector at CDTECH | Inventor | Manufacturer
5moAWS's Bedrock enhancements leveraging prompt routing and caching demonstrate a sophisticated understanding of latency reduction through intelligent query distribution. The integration of advanced RAG evaluation metrics within Bedrock signifies a move towards more robust and measurable AI performance assessment. How would you adapt the SPDL framework to optimize data loading for large-scale multimodal training datasets in a federated learning environment?