Hugging Face’s Post

In times of hype, run your own experiments. How? Use Hugging Face Inference Providers With every new open model release, social media timelines are full of contradicting information, exaggerated claims, etc. That's why running quick (and cheap) experiments is becoming critical. - Have you heard that the latest Llama 4 models are bad? - Have you heard that Llama 4 models behave differently across providers? - Is QwQ-32B better than DeepSeek R1? Run these models through data you care about. With the Hub you can: - Get access to the latest models (from Day 0). - Test them even if you don't have GPUs. - Mix and match the fastest, most reliable inference providers. - Discuss and learn about these models with the largest AI community The prompt and results in the image attached are part of "vibench" a tiny benchmark I'm building with Inference Providers. It contains interesting and challenging prompts from Reddit, the Sparks of AGI Microsoft's paper, and other places. You can find the open dataset in the first comment, and feel free to suggest challenging prompts to add them to vibench. and misinformation, run your own experiments. How? Use Hugging Face Inference Providers With every new open model release, social media timelines are full of contradicting information, exaggerated claims, etc. That's why running quick (and cheap) experiments is becoming critical. - Have you heard that the latest Llama 4 models are bad? - Have you heard that Llama 4 models behave differently across providers? - Is QwQ-32B better than DeepSeek R1? Run these models through data you care about. With the Hub you can: - Get access to the latest models (from Day 0). - Test them even if you don't have GPUs. - Mix and match the fastest, most reliable inference providers. - Discuss and learn about these models with the largest community. The prompt and results in the image attached are part of "vibench" a tiny benchmark I'm building with Inference Providers. It contains interesting and challenging prompts from Reddit, the Sparks of AGI Microsoft's paper, and other places. You can find the open dataset in the first comment, and feel free to suggest challenging prompts to add them to vibench.