Prolific

Introducing AI Task Builder: The fastest way to train and evaluate AI 🦾 We’re excited to launch early access to our new product—AI Task Builder. Built API-first and powered by the Prolific platform, AI Task Builder helps developers and researchers build better #AI products faster, programmatically or through our intuitive interface. Learn more and request early access ➡️ https://lnkd.in/eZDbxXSy #AITools #AIResearch #AITaskBuilder

Behind every major scientific advancement lies countless human contributions. Some participants translate languages, others identify galaxies, or many simply share their daily experiences. In this short article series, we’re spotlighting 5️⃣ notable breakthroughs across science, technology, and society. This edition features Mozilla Firefox, UNICEF, reCAPTCHA, and more. 📰

How are you staying ahead in the AI market? ⚔️ Prolific VP of Product Sara Saab predicts that more people will apply their own values to open-source models, so they can diversify their products in the race to outperform the rest. #AI #AIDevelopers #Technology

As LLMs continue to evolve in sophistication and capability, the methods we use to evaluate their performance need to keep up. Among these methods, the "LLM-as-a-judge" concept has emerged. How effective is this technique overall? More importantly, how does human feedback factor into creating effective evaluation frameworks? In this feature, we explore the technical underpinnings, practical applications, and critical challenges of the LLM-as-a-judge methodology—with particular attention to how Prolific is bridging the gap between automated evaluation and essential human judgment. 🔗 https://lnkd.in/eYmhhxqz

This week’s Findings in Four is about the impact of GenAI on critical thinking, with research led by Carnegie Mellon University and Microsoft Research. 319 diverse knowledge workers. 936 detailed examples of real AI use cases. Here’s how recruiting taskers from Prolific resulted in rich data on how self-confidence shapes cognitive engagement ➡️ 🗞️ See the full summary: https://lnkd.in/ehwHc73v

Demoing AI Task Builder with Adam Winning ✨ You can now play back the full product demo of AI Task Builder from our “Evaluating AI applications with human feedback” webinar! AI Task Builder is the fastest way to train and evaluate AI. Built API-first, this exciting product enables you to automate and accelerate your text annotation tasks in one integrated workflow. 💻 Watch now: https://lnkd.in/eUNgftey #AI #ProductDemo

Enjoyed discussing the future of the human workforce behind AI with The Economist. While headlines often focus on automation, the shift that's happening is more nuanced: the type of human input needed for AI is evolving. Basic labelling is shifting towards nuanced evaluation, ethical testing, and specialised feedback – tasks requiring diverse skills, cultural understanding, and deep expertise. This creates new opportunities for people globally to shape the impact of technology on their lives. This is the "Human Intelligence Layer" in action. As I mentioned in the piece, Prolific is building the human data platform to support these challenges. We provide reliable access to the diverse, high-quality human perspectives essential for developing algorithms that are not just powerful, but also nuanced, fair, and aligned with real-world needs. Read the full article here: https://lnkd.in/ePhtDA4u Interested in helping train, and evaluate AI? Come join us! https://lnkd.in/e6WQ44Wr

Which base model are you choosing? ⚒️ There are plenty of readily available LLMs that you can use as a base and fine-tune to your needs—but that’s the dilemma. Base models have different license types, sizes, guardrails, language capabilities, and more to consider. Besides the generalist range, there are also several domain-specific models that enterprises and app development teams can look at. We cover it all in our practical guide to building AI apps with LLMs. 📥 Download it here: https://lnkd.in/egVJQTHf #AI #LLMs #AIApplications

Your AI evaluations and research just got easier. We’ve rolled out improvements across Taskflow, UI, plus added new features you’ll love. Our latest #WhatsNew update includes: 💻 Taskflow performance upgrades: Collect annotations of larger datasets faster than before. 🔢 Quota sampling by age range: Ensure balanced representation and more. 👥 Editable participant groups: Organize your data collection projects with clear, consistent naming conventions. See the details 👉 https://lnkd.in/eaczbyaP

When researchers become participants 🤝 We love to hear from participants about how they started on Prolific and what sets it apart. Esmeralda, who’s been a participant for two years, talked to us about how she used Prolific for her academic research before realizing she could earn high rewards for her knowledge.