TensorFuse (YC W24)

New blog alert 📖 - Understanding Multi GPU Communication and Nvidia NCCL for finetuning models. Recently, one of our users were fine-tuning LoRA adapters via Axolotl. They ran into an issue where some occasional training jobs would run extremely slowly and eventually crash with a “Watchdog timeout” error. So we dig deep into the Nvidia NCCL rabbit hole, fixed the issue and wrote a blog about it. In this post, you’ll learn: - What NCCL does and why it’s critical for multi-GPU training - How we fixed one of the most common challenges of Nvidia’s NCCL library - the dreaded “watchdog timeout” error. Read the full blog here: https://lnkd.in/dCwY-a2d Let us know in the comments if you ever run into similar issues and how did you fix them.

A user recently signed up on Tensorfuse after they failed to deploy their custom models on Sagemaker. They had custom bio models which they wanted to run on Sagemaker Serverless but couldn't get it running even after writing custom code just to make their models compatible with Sagemaker SDK. They also ran into the Image size limit on Sagemaker Serverless which allows to run images only upto 10GB in size. After signing up on Tensorfuse, they were able to deploy not just one but 5-6 models on Serverless GPUs directly on their AWS account. All within a single day! And Tensorfuse charges just 10% premium on top of EC2 as compared to 40% by Sagemaker. Better, faster and cost effective. If you running serverless AI inference on AWS, get started with Tensorfuse. Link in the comments!

New milestone unlocked! Product so good, people tell their friends about it. Recently a user signed up our platform and within just a couple of hours, they were able to deploy the latest Qwen QWQ model on production ready infra that could scale to 1000s of concurrent request. All they had was a Dockerfile and an AWS account. If you want to run serverless AI inference on your AWS, the developer experience TensorFuse (YC W24) offers is 100x better than alternate solutions like Sagemaker or Bedrock. In fact, it's so good, our users share about a dev tool to their friends like its a Netflix movie.

⚡️Here’s the ultimate guide to fine-tune any LLM into your reasoning model! Highlights of the guide (find the guide in the comments): • Fine-tunes Qwen7B with GRPO (DeepSeek’s RL algo) + Unsloth for faster training. • Deploys a vLLM server for inference on AWS (L40S). • Saves the LoRA adapters to Hugging Face for easy reuse. Start building smarter LLMs with Tensorfuse! Sign up via the link in the comments for swift deployments👇

Deploying GenAI models shouldn't feel like holding up a collapsing building ☠️ Your time, budget, and bandwidth shouldn't be sacrificed to deployments (and you shouldn’t be babysitting servers on weekends). With Tensorfuse, all you need is a CLI command to deploy your GenAI model so you can focus on building great models while we manage the infrastructure. Get started with Tensorfuse now for seamless deployments! Find the link in the comments 👇

Alibaba’s latest AI model, Qwen, is the new talk in town 🔥 This compact model (only 32 billion parameters) rivals cutting-edge reasoning models like DeepSeek-R1 (670B) & OpenAI o1-mini in mathematical benchmarks and scientific reasoning tasks. ⚡️ Try deploying this pocket-sized model with Tensorfuse on AWS <8 min. Find the link to the guide we just released in the comments 👇

Day 1 at the NVIDIA GTC conference! If ever there was a developer concert, this would be it!

🚀 Deploy DeepSeek-R1 in the most efficient way with Llama.cpp using TensorFuse (YC W24) If you have tried to deploy large LLMs like DeepSeek-R1, there’s a high possibility that you've struggled with GPU memory bottlenecks. We have prepared a guide to deploy LLMs in production on your AWS using Tensorfuse. What’s in it for you? • Ability to run large models on economical GPU machines (DeepSeek-R1 on just 4xL40s) • Cost-Efficient CPU Fallback (Maintain 5 tokens/sec performance even without GPUs) • Step-by-step Docker setup with llama.cpp optimizations • Seamless Autoscaling Skip the infrastructure headaches & ship faster with Tensorfuse. Find the Guide in the comments 👇

Another day without GPU autoscaling? Managing cloud infra can be a huge time and cost sink but it doesn’t really have to be. With Tensorfuse, you can focus on building and coding while we handle the complexities of infrastructure management on your own cloud. Simplify your cloud journey with Tensorfuse today! Link in the comments 👇

⚡️ Exciting Announcement: Tensorfuse is now part of the NVIDIA Inception Program! With this collaboration, we're accelerating our mission to become your go-to solution for: • Fine-tuning AI models to your exact needs • One-click deployment on your own cloud infrastructure • Intelligent autoscaling that adapts in real-time Catch us at NVIDIA GTC 2025 (March 17-21, San Jose) to discuss all things GenAI!