Anyscale

THIS WEEK: Anyscale will be at Google Next in Vegas, April 9-11🔥 Come see us at booth #1702 to learn how Ray Turbo + GKE power the distributed OS for AI 🙌 High performance 🚀 | Low cost 🤑 | Massive scale 💪 If you're pushing the limits of AI infra, stop by and talk to us about how Ray Turbo can accelerate your AI infrastructure → https://lnkd.in/gMF2PAn5

How do Ray and Kubernetes relate? This is one of the most common questions we get, and there's a crisp explanation. Kubernetes is built for platform engineers, people who live and breath YAML and containers, who operate fleets, manage permissions, monitor costs, standardize logging and observability tooling. Ray is built for machine learning people, people who write Python code, who live and breath PyTorch and NumPy and Hugging Face, who think about batch sizes, learning rates, and embedding computations. Each on their own misses part of the picture. Together, they form a software stack for AI that addresses both sets of needs.

Scaling AI is hard – but you’re not alone. 🙌 Join us at the Ray + AI Infra Summit (May 20, NYC🗽), where engineers from Character.AI, Spotify, Figma, Bridgewater Associates, and more will share how they’re building fast, reliable, and cost-efficient systems at scale. 🚀 Learn how teams are:  • Managing distributed workloads with Ray  • Reducing training time and infra spend  • Designing systems that scale with product growth Whether you're deep in production or just getting started, this event delivers practical insights you won’t find anywhere else. Reserve your seat → https://lnkd.in/gETcMReC

Uber built a unified ML platform that abstracts away infra complexity — letting teams run Ray jobs without worrying about clusters or resource placement. Ray + Kubernetes handle orchestration and scaling across Uber's fleet. 🤝 Full setup breakdown 👇 https://lnkd.in/gARjDJsN

Anyscale Public Courses Training is here! 🙌 Join us next week April 7-8 for Introduction to Ray and Anyscale, a hands-on course covering model training, HPO, data processing, and scalable deployment with Ray. 🔥 Session is guaranteed to run Register now 👉 https://lnkd.in/gsW4xZQb

🚀 In Ray 2.44, we're giving Ray a major upgrade for scaling LLM inference. We're seeing a ton of companies and users organically using Ray with vLLM to scale LLM serving and batch LLM inference. In theory, the two technologies are very complementary -- vLLM provides best-in-class performance for LLM inference, and Ray is the defacto way for AI infrastructure teams to scale inference workloads. But previously, in order to do this you'd need to write a lot of boilerplate to make your LLM inference performant at scale. In the most recent Ray release, we've launched Ray Data LLM and Ray Serve LLM. These APIs allow for simple, scalable and performant ways of deploying open source LLMs as part of existing data pipelines and Ray Serve applications. In the near future, we'll be working on building out more examples and reference architectures for deploying key models like Deepseek on Ray + vLLM (and SGLang support as well!) Check out our blog for more details: https://lnkd.in/gNhg2BRU

Anyscale will be at Google Next in Vegas, April 9-11! AI at scale needs the right foundation 🚀 Visit us at booth #1702 to see how Ray Turbo, the AI compute engine from Anyscale, and GKE power the Distributed OS for AI – delivering top performance, cost efficiency, and scalability. Let’s talk about how Ray Turbo can accelerate your AI workloads. See you there! 👋 https://lnkd.in/gbbwgFXy

Join us on April 17 for the next Ray Meetup, co-hosted with LanceDB. 🚀🙌 Hear from LanceDB Lei Xu will share how LanceDB helps LLMs run on a single source of truth, while Anyscale's Richard Liaw covers how to deploy and scale LLM workloads in production with Ray. Spots are limited, sign up now → https://lu.ma/u0cjfsqo

NEW IN RAY: Native LLM APIs for Ray Data and Ray Serve Libraries 🙌

The Ray + AI Infra Summit (NYC🗽, May 20th) offers something truly special: direct access to the brilliant minds who created the technology. This isn't just another tech talk – it's a master class from the pioneers who are defining the future of AI infrastructure. Whether you're already using Ray or exploring solutions for scaling your AI workloads, this summit provides insights you won't find anywhere else. Limited seats available, save your spot today → https://lnkd.in/gETcMReC