Elder Research

AI is increasingly handling the “easy” decisions for us—optimizing prices, managing supply chains, and personalizing customer experiences. Companies like Amazon have taken a hands-off approach, letting algorithms drive entire business functions. Our best algorithms have purposefully limited information, which makes them consistent and useful but also means they may miss unusual circumstances that a person knows about. As AI takes on more decisions, who’s making sure those decisions are safe, fair, and optimized for the business? At Elder Research, we’ve spent three decades helping organizations navigate this challenge. And recently, we pulled together everything we’ve learned into our Responsible AI Framework. It’s a starting point for what we hope will become productive conversations at organizations developing AI solutions. Because we know solutions that are designed responsibly make a greater impact in the long run. And at the end of the day, responsible AI has to consider both the governance and technical aspects of managing AI. AI governance provides a framework to consider and manage enterprise risk. Executives and boards care about regulatory compliance, reputation, and accountability because the enterprise is liable for all AI decisions. The technical aspects of responsible AI deal with the fairness of the input data, lineage of how data is incorporated into decision processes, model validation, and model monitoring. The key question isn’t just “Can we automate this?” but “Should we?” and “How do we remain in control?” AI should accelerate decision-making—not remove human oversight. Over the next few weeks, I plan to share a few thoughts on both the business risks and the technical challenges of building AI systems we can trust.  How is your organization thinking about AI risk and governance? #ResponsibleAI #RAI #AI

Indianapolis is buzzing, and so are we. 🐝 We’re on ground at the INFORMS Analytics+ Conference! Our team is excited to help facilitate sessions, moderate a panel, and connect with folks at our booth. And yes … we brought coffee! ☕ If you’re at the conference, stop by Booth 307 to say hi and enter our 𝗖𝗼𝗳𝗳𝗲𝗲 𝗕𝗿𝗲𝗮𝗸 𝗚𝗶𝘃𝗲𝗮𝘄𝗮𝘆: fresh-roasted coffee from one of our favorites, a mug, and two great reads—𝘔𝘪𝘯𝘪𝘯𝘨 𝘠𝘰𝘶𝘳 𝘖𝘸𝘯 𝘉𝘶𝘴𝘪𝘯𝘦𝘴𝘴 and 𝘙𝘦𝘴𝘱𝘰𝘯𝘴𝘪𝘣𝘭𝘦 𝘋𝘢𝘵𝘢 𝘚𝘤𝘪𝘦𝘯𝘤𝘦. We’re here for the insight, the ideas, and the conversations. Big thanks to the INFORMS team for hosting another great conference! #2025Analytics #DataScience #DataAnalytics

PyData Virginia 2025 is happening April 18 and 19—and we’re excited to be part of it! Among many great sessions, our very own Josh Fairchild and Liam Agnew will be speaking on how a “Data as a Product” mindset can elevate client satisfaction and long-term business value. This conference is an amazing opportunity to dive deep into the world of data science, learn about the latest technologies, and meet an amazing community of professionals and enthusiasts. We’d love for you to join us! Learn more and save your spot at pydata.org/virginia2025. Whether you’re a Python pro or just getting started with data, there are sure to be some valuable takeaways. We’re proud to sponsor this year’s event, hosted by PyData Global, and grateful to join the conversation! #PyDataVirginia25 #PyData #NumFOCUS #Python #DataScience

A few interesting articles shared across Elder Research last week! 1. “Pen and Paper Exercises in Machine Learning”: https://lnkd.in/dYpJTfys. With all the focus on LLMs and all the bytes written about the computers doing everything for us soon, I’m more grateful than ever to find resources like this (from 2022). This also seems like a useful instrument for deliberate practice (https://lnkd.in/dqA2E79A)! 2. Image Generation with OpenAI's 4o model: https://lnkd.in/dSb89BHn. I’ve been paying for ChatGPT Plus lately to get a better sense of when and where these chat interfaces are useful to me. This update to 4o is a remarkable improvement over my previous experiences (presumably with DALL-E). I haven’t found many applications for generated images yet that aren’t over the top or in poor taste, but when they turn up—this looks like a good solution. 3. “Tracing the thoughts of a large language model”: https://lnkd.in/dUSmj8-Z. More interesting stuff from Anthropic, working to understand what LLM computations look like more or less in vivo. There’s an interesting duality here: these models are fundamentally super simple, but at the “macro” level their operations are very difficult to understand or reason about.

What happens when bright students and data professionals get together? Lots of great questions, honest conversation, and a few aha moments for everyone involved. 💡 It was amazing to recently have another group of University of Virginia students visit our Charlottesville office to learn what it’s like to work in data consulting. Our founder, John Elder, kicked things off with the story of how our company began and grew over time. Then our team shared their own stories—what it’s like to be a data scientist, engineer, technical business analyst, or client engagement manager at Elder Research. 🧑💻 The students came with some great questions, and our team really enjoyed the chance to share more about the work we do, how we do it, and why it matters. The day ended with some great food and even more great conversation at dinner. 🍽️ We’re excited to see where these talented students go next. A big thank you to Professors Karen Schmidt and Xin Cynthia Tong for helping to bring this gathering to life!

Scaling is an underappreciated trick in forecasting. But don’t take my word for it. The winners of the m5 forecasting uncertainty competition said this: “Time-series data often span several orders of magnitude, requiring scaling and normalization to generalize across related series.”[1] Scaling helps a model more easily learn patterns that appear in different series on different scales. It does this by converting different time series to be on similar levels. (This is only relevant for models that are trained on many time series at once) For example, cold medicine sales go up during the winter. By how much? The pattern at larger stores could help a model forecast how much they’ll go up at smaller stores. Let’s say in a large store there’s a 3X increase. This could serve as a baseline for a smaller store that’s more prone to random variation. Normalizing the data to be on the same scale makes identifying these patterns easier for a model. But how should you scale? Thinking and doing a little research, Kimberly Keiter and I came up with a few different ways: 1 – Standardization: subtract the mean, divide by standard deviation. 2 – Min-max normalization: convert all values to between 0 and 1 based on each series min and max. 3 – Divide by the mean: Divide each series by its mean, so that the average value is 1 for each series. 4 – Divide by an aggregated mean per time period: for example, if we’re forecasting daily store-product sales, divide sales by the average sales at that store during that day. Used by the m5 uncertainty winners. In general, scaling reduces variance but increases the bias of a forecasting model. It’s certainly possible to over-scale: compress time series that act differently onto the same values. This is akin to underfitting. I tend to think methods 1 and 2 usually fall into this category. Differencing is worth mentioning here too: predict the difference between values instead of the values themselves. Not really a scaling method, but it’s a similar idea. More common in univariate methods. What methods did we miss? What have you had success with? Let me know! [1] https://lnkd.in/eXuKGxyy

There are lots of jobs out there, but this helps make them great: Getting to know people beyond their job descriptions. That’s one of the things our CEO Gerhard Pilcher mentioned as a core value while chatting with our newest team members. Every time new people join our crew, our CEO, COO Jeff Deal, and founder John Elder love spending time getting to know everyone and sharing the vision for the work we do. 👋 Join us in welcoming our new team members, and learn a little more about them! 𝗦𝗽𝗲𝗻𝗰𝗲𝗿 𝗔𝗹𝗹𝗴𝗮𝗶𝗲𝗿, 𝗗𝗮𝘁𝗮 𝗦𝗰𝗶𝗲𝗻𝘁𝗶𝘀𝘁: enjoys coaching soccer and playing music (currently learning piano) 𝗔𝗺𝘆 𝗖𝗮𝗿𝘂𝘀𝗼, 𝗦𝗲𝗻𝗶𝗼𝗿 𝗠𝗮𝗻𝗮𝗴𝗲𝗿, 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 & 𝗗𝗲𝘃𝗲𝗹𝗼𝗽𝗺𝗲𝗻𝘁: loves investing in people’s learning and traveling to new places 𝗬𝗼𝗹𝗮𝗻𝗱𝗮 𝗣𝗶𝗻𝗰𝗸𝗻𝗲𝘆, 𝗦𝗲𝗻𝗶𝗼𝗿 𝗗𝗮𝘁𝗮 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿: has been smiling since she joined the team and is a proud mom of her grown daughter 𝗝𝗮𝗲 𝗟𝗲𝗲, 𝗗𝗮𝘁𝗮 𝗦𝗰𝗶𝗲𝗻𝘁𝗶𝘀𝘁: focused on parenting as she celebrates recently having her second child 𝗬𝗮𝗿𝗮 𝗥𝗶𝗼𝘀-𝗤𝘂𝗶𝗿𝗼𝗴𝗮, 𝗗𝗮𝘁𝗮 𝗔𝗻𝗮𝗹𝘆𝘀𝘁: loves reading, trying different foods, playing video games, and taking care of her guinea pig 𝗦𝗵𝗮𝗻𝗲 𝗙𝗿𝗮𝗹𝗲𝘆, 𝗗𝗮𝘁𝗮 𝗦𝗰𝗶𝗲𝗻𝘁𝗶𝘀𝘁: was a math teacher and loves all things nerdy 𝗭𝗮𝗿𝗮𝗳𝘀𝗵𝗮 𝗨𝘇𝘇𝗮𝗺𝗮𝗻, 𝗗𝗮𝘁𝗮 𝗦𝗰𝗶𝗲𝗻𝘁𝗶𝘀𝘁: likes visiting coffee shops, reading, and exploring walking trails What are some things you think make for a good workplace? If you’re interested in joining our team or know someone else who might be, check out our career opportunities: https://lnkd.in/gpPJUkTT #CareerOpportunities #DataScience #DataAnalytics

We believe AI built with integrity unlocks its full potential. That’s why 100% of our team—technical and nontechnical—completes our internal responsible AI (RAI) course. 🧑💻 AI should be unbiased, reliable, and built with accountability in mind. By embedding these principles into everything we do, we ensure AI is not only trustworthy but also a driver of breakthrough innovation. Because at the end of the day, the best solutions are the ones people trust—and actually deliver dependable results. What’s your team doing to keep AI solutions on track? 💡 Explore the principles of our RAI framework: elderresearch.com/rai #ResponsibleAI #AIinnovation #TrustworthyAI #AILeadership

March Madness is here, and it’s time to rethink your bracket strategy! 🏀 Robert Robison says there's a BIG flaw with bracket pools: they incentivize safe picks (no upsets), making things less exciting and strategic. Some have tried to combat this by adding an upset bonus—a reward for correctly picking upsets. ⛹️ But Robert says there’s a challenge with the typical upset bonus: “It’s either not large enough to matter or so large that it’s senseless to pick anything but an upset.” Check out Robert’s blog for insights on how a different approach to the upset bonus can create a fairer scoring system, where both risky and safe picks are valued, making your bracket more exciting and competitive. 𝗥𝗼𝗯𝗲𝗿𝘁 𝗯𝗿𝗲𝗮𝗸𝘀 𝗶𝘁 𝗮𝗹𝗹 𝗱𝗼𝘄𝗻 𝗵𝗲𝗿𝗲: https://lnkd.in/eEuNEYzj #MarchMadness

Elder Research analytics link roundup, March 10–14, 2025: What does “overfitting” mean, anyway? https://lnkd.in/eaArJ-Tc 1. “Use a test set to select among the models that fit your training data well. It’s not that complicated.” Part of a larger series on Ben Recht’s blog that sparked good discussion. https://lnkd.in/eqkKVHCa 2. Simon Willison’s recent review of large language models in 2024. I’m reading more and more of this recently—and trialing ChatGPT Plus and Cursor. I'm still mostly ambivalent about these things. https://lnkd.in/e8sDb6Bj 3. Check out these wild graphs showing how StackOverflow queries for things like “R” and “Pandas” have been wiped out in the ChatGPT era. I wonder where this all ends up. Robert Robison has the right take, I think, which is to wonder where the equilibrium state is: Are we going to get to a place where LLMs totally replaces StackOverflow, etc.? Will we find a spot where LLMs can answer the easy questions, leaving us to talk to actual humans for the harder ones? Or (I fear), will LLMs put StackOverflow out of business and then we won’t have a centralized place for these kinds of discussions anymore? https://lnkd.in/est_jF3p 4. “The State of Machine Learning Competitions, 2024 Edition”: I was surprised at the number of competitions and the total prize sizes. A pretty healthy variety of solution types, too; it isn’t all deep learning all the time. https://lnkd.in/ejv2KdzG