From the course: Responsible Generative AI and Local LLMs

Unlock this course with a free trial

Join today to access over 24,800 courses taught by industry experts.

Invoke an LLM on an AWS G5 instance, part 1

Invoke an LLM on an AWS G5 instance, part 1

From the course: Responsible Generative AI and Local LLMs

Start my 1-month free trial Buy for my team

Invoke an LLM on an AWS G5 instance, part 1

“

- So this week, I'm going to dive in to a interesting scenario here where you can use the package management system in Rust to invoke large language models. So I think everybody's talking about large language models and how cool they are, but what's interesting is you'll see people, you know, giving these really complex workflows, applications, all this, you know, complexity around running a large language model. But because of the beauty of Rust and the Cargo package management system, you don't have to do anything. You can just actually use the Cargo system to invoke models. I'm going to do this with Hugging Face Candle and I'm going to do it on an AWS GPU that's a very powerful GPU. So let's go ahead and take a look at how we would do that. So first up here, I've got Hugging Face Candle. Let's just look at what this does. So this is a minimalistic ML framework for Rust. And you can see the code is really simple, right? But the biggest thing here, and I think this is why it's so…

Contents