System 1 and System 2 Thinking in AI: A Framework for Modern Market Research

Theory of the mind and the workings of our brains have captivated me for many years.  There are so many fascinating aspects of how we think, and how our minds evolve to study and understand.  I was recently watching a YouTube video that was revealing insights from studies on the minds of infants and children.  Some of the points the video made included these.  A 12-month-old baby can already recognize that an object continues to exist even when hidden from view. By 18 months, they understand that others may have different beliefs than they do. By age 4, children develop sophisticated theories about other people's thoughts and intentions.

The human mind, even in its earliest stages, demonstrates remarkable capabilities that we're only beginning to understand.  This innate cognitive architecture which effortlessly blends intuition with reasoning, emotion with logic has been the subject of extensive research. Perhaps most famously, psychologist Daniel Kahneman described our thinking as operating in two distinct modes: System 1 (fast, intuitive, automatic) and System 2 (slow, deliberate, analytical). This framework has influenced our understanding of decision-making, from consumer behavior to economic policy.

As generative AI transforms market research, this same framework offers a surprisingly useful lens for understanding different AI models and their optimal applications, helping us align artificial capabilities with the natural cognitive processes they aim to simulate.

The AI Cognition Spectrum

Modern generative AI models exhibit characteristics that parallel human cognitive processes in intriguing ways:

"System 1-like" AI Models (standard GPT models like gpt4/4o, Claude 3):

• Generate responses quickly and fluently
• Excel at pattern recognition and associations
• Process information holistically
• Operate without explicitly showing their reasoning
• Can be prone to shortcuts and biases

"System 2-like" AI Models (reasoning-optimized models like Claude 3.5 / Sonnet and OpenAI o1 or 03):

• Work through problems step-by-step
• Exhibit more explicit logical processes
• Are better at catching and correcting errors
• Are less prone to jumping to conclusions
• Can more effectively override initial impressions

This analogy is not perfect but it provides a useful framework for matching AI capabilities to specific research needs.

Ongoing Exploration and Frameworks

At Bellomy, our team recognizes that we're in the early stages of understanding how to optimally leverage these different cognitive modes in AI. Rather than claiming to have all the answers, we're engaged in a deliberate journey of exploration and learning.

Our approach goes beyond simply "having a synthetic panel” or using GenAI.  We are developing systems that leverage the distinctions between different AI cognitive styles. Through ongoing testing of synthetics across various research applications, from standard surveys to AI chat interviews, and cross-validation with real human responses, we're establishing crucial decision points that aren't immediately obvious but prove essential for optimal outcomes.

This continuous learning process has yielded valuable insights and practices that guide our implementation. We're building systems that know when to use the right model with the right settings for specific research tasks, eliminating confusion for end-users while still delivering powerful results.

We view this as a collaborative journey and actively seek partnerships with forward-thinking organizations interested in exploring these capabilities together. By combining our growing expertise in AI implementation with partners' domain knowledge and research needs, we can accelerate learning and develop more robust solutions.

Applications Across the Research Spectrum

Data Analysis: The Right Brain for the Right Insight

System 1-like AI excels at quickly identifying patterns in large datasets, generating initial hypotheses, and creating compelling narrative summaries. These models can rapidly process information and suggest unexpected connections.

System 2-like AI shines when deeper causal analysis is needed, particularly for evaluating statistical significance, identifying potential confounding variables, and methodological critique. These models are more transparent in their analytical reasoning.

Our testing shows that the optimal approach often involves a hybrid: using System 1-like models for initial exploration and hypothesis generation, then System 2-like models for rigorous validation and causal inference.

Qualitative Research: Balancing Engagement and Discipline

In AI-moderated qualitative research, the distinctions become particularly important:

System 1-like AI creates natural conversational flow, adapts quickly to unexpected participant responses, and demonstrates the empathetic qualities that build rapport. These models keep participants engaged and conversations flowing.

System 2-like AI maintains disciplined adherence to discussion guides, systematically probes for deeper insights, and ensures comprehensive coverage of research objectives. These models excel at tracking complex interview structures.

Our implementation at Bellomy pre-configures the optimal model and settings based on the specific qualitative objective, eliminating confusion for researchers while delivering superior insights.

Synthetic Respondents: Mirroring Human Decision Processes

Perhaps the most fascinating application is in synthetic respondent creation and use:

System 1-like AI creates more natural responses that capture emotional nuance and the associative nature of many consumer decisions. These models excel at generating the type of quick, instinctive reactions that drive many purchase decisions.

System 2-like AI produces more consistent and logical synthetic responses, maintaining alignment with established behavioral models and demographic profiles. These models better handle complex decision trees with multiple variables.

Choice-Based Exercises: Mimicking Complex Human Decisions

In choice-based methodologies like conjoint analysis, real human decisions involve both cognitive systems: System 1 for initial reactions to options and attributes, System 2 for weighing tradeoffs and making final selections.

Our testing at Bellomy has revealed that simulating these decisions requires a carefully calibrated approach using different AI models at different stages of the choice process:

1. Initial Attribute Evaluation: System 1-like models better capture intuitive preferences and emotional reactions to product features
2. Tradeoff Analysis: System 2-like models better handle the rational evaluation of multiple attributes and complex tradeoffs
3. Consistency Maintenance: System 2-like models ensure logical consistency across multiple choice scenarios

By carefully orchestrating these capabilities, we are developing synthetic respondent systems that reflect the complex, multi-layered nature of human decision-making.

Implementation Insights: What We've Learned

Through our testing and validation, our team has identified several crucial factors that aren't immediately obvious but dramatically impact the quality of AI-generated insights:

1. Context Window Management: The amount of relevant information provided to the model significantly influences response quality, but this optimal amount varies by task and model type
2. Temperature Calibration: Finding the right balance between deterministic responses (low temperature) and more creative exploration (high temperature) depends on both the research objective and the model type
3. Model-Specific Prompt Engineering: Different models respond differently to the same prompts, requiring tailored approaches for optimal results
4. Cross-Validation Frameworks: Establishing reliable benchmarks for evaluating synthetic responses against human responses requires sophisticated methodological approaches

We are approaching these challenges by pre-building and pre-testing prompts, selecting optimal models and settings for specific research tasks, and creating user interfaces that shield researchers from these technical decisions while still delivering superior results.

The Future: Orchestrated Intelligence

Looking ahead, we see the most promising approach as neither System 1 nor System 2 in isolation, but rather an orchestrated intelligence that leverages each model type for what it does best. This mirrors how human cognition works, with constant interplay between intuitive and analytical thinking.  While hybrid models are emerging from the GenAI vendors, there is a market research need that remains relevant and is not automatically met by such models.

I firmly believe that the organizations that will gain the greatest competitive advantage won't be those that simply adopt AI, but those that understand the nuanced capabilities of different AI approaches and strategically deploy them within well-designed research frameworks.

At Bellomy, we're committed to continuing this exploration and refinement, developing increasingly sophisticated implementations that combine the best of human research expertise with the expanding capabilities of AI cognition across the spectrum from System 1 to System 2.

Conclusion: A New Framework for AI in Research

Understanding AI capabilities through the System 1/System 2 lens offers market researchers an important framework for navigating the rapidly evolving landscape of generative AI. Rather than treating AI as a monolithic capability, this approach recognizes the diverse cognitive styles that different models embody and brings us closer to matching human thinking.

By matching these capabilities to specific research needs and implementing them with carefully calibrated settings, prompts, and frameworks we can move beyond the basic question of "whether to use AI" to the more sophisticated question of "which AI approach will best serve this specific research objective."

The future belongs not to those who simply adopt AI, but to those who understand its nuances and deploy it strategically within well-designed research methodologies.

References:

Piaget, J. (1954). “The construction of reality in the child”

Onishi, K. H., & Baillargeon, R. (2005). “Do 15-month-old infants understand false beliefs?”

Wellman, H. M., Cross, D., & Watson, J. (2001). “Meta‐analysis of theory‐of‐mind development: The truth about false belief.”

Kahneman, D. (2011). “Thinking, fast and slow.”