Understanding Function Calling in AI

Just got back from the Microsoft AI Tour in NYC last night! It was great to connect with peers, sharing insights and collaborating not just within health and life sciences, but across industries.

One key takeaway was Microsoft’s announcement that DeepSeek R1 is now hosted on Azure, a topic that came up frequently in discussions. The excitement around this model really ties to Sustainability in AI as it promises the potential of reducing the cost and energy consumption of training and inference operations. Of course there are concerns about a privacy and security with a model trained in China but private model-as-a-service hosting on Azure makes it easier to experiment securely, without compromising data privacy.

The biggest theme overall is of course that this is the year of AI Agents. While nearly every company is marketing this concept, many of the products really are just rebranded chatbots.

In this article, I'll unpack the key enabling technologies that make agentic AI real so that you can distinguish the hype from the innovation.

Structured Outputs and Function Calling in AI Models

One of the most critical enablers of Agents is structured outputs—a key component of function calling in modern foundation models.

Structured outputs allow foundation models to interact with external systems in a predictable, machine-readable way. Instead of generating free-form text, models output structured data (typically in JSON format), specifying what function to execute and its required parameters. This ensures seamless automation with APIs, databases, and workflows.

Several advanced AI models support structured outputs and function calling, including:

• GPT-4o (OpenAI) – Enables structured JSON outputs for API integrations.
• Claude 3 (Anthropic) – Supports structured reasoning and function execution.
• Gemini 1.5 (Google DeepMind) – Facilitates structured outputs for tool use and API interactions.
• Mistral Large (Mistral AI) – Implements structured data handling for controlled function execution.

This capability transforms AI from a passive responder to an active orchestrator, enabling models to retrieve real-time data, trigger external actions, and make decisions dynamically.

Defining Functions for a Foundation Model

To make this work, developers define callable functions using JSON schema. Let’s look at a real example of how GPT-4o can execute function calls.

Imagine you need real-time weather data and the ability to schedule meetings. You could define these functions like this:

[
  {
    "name": "get_weather",
    "description": "Retrieve the current weather for a location.",
    "parameters": {
      "type": "object",
      "properties": {
        "location": {
          "type": "string",
          "description": "The city and country, e.g., 'Boston, MA'."
        }
      },
      "required": ["location"]
    }
  },
  {
    "name": "create_meeting",
    "description": "Schedule a meeting in the calendar.",
    "parameters": {
      "type": "object",
      "properties": {
        "title": { "type": "string", "description": "Meeting title." },
        "date_time": { "type": "string", "format": "date-time", "description": "Start time in ISO format." },
        "attendees": { "type": "array", "items": { "type": "string", "format": "email" }, "description": "Attendee emails." }
      },
      "required": ["title", "date_time", "attendees"]
    }
  }
]        

By defining functions this way, the model knows exactly what actions it can request, enabling seamless integrations with external systems.

How the Orchestrator Executes Function Calls

Once the model generates a function call, a software orchestrator steps in to execute the request. The orchestrator could be:

• A client application (example Chat GPT)
• An automation platform (example Power Automate)
• A backend service running AI logic

Step-by-Step Execution:

1. Model Generates Structured Output If a user asks: "What's the weather in Boston right now?", GPT-4o returns:

{
  "name": "get_weather",
  "arguments": { "location": "Boston, MA" }
}        

2. Orchestrator Executes the Function

3. Returning Results to the Model The API responds:

{
  "temperature": "32°F",
  "condition": "Snowing",
  "humidity": "78%"
}        

4. Model Interprets and Responds to the User GPT-4o converts this into natural language:

"The current temperature in Boston is 32°F with snow, and humidity is at 78%."

This structured approach ensures AI can request data, process results, and dynamically update responses in real-time.

Advanced Use Cases and Multi-Step Execution

Function calling enables complex workflows, such as:

• Chained Execution: A model can call multiple functions in sequence (e.g., fetch a stock price, then send an alert if it drops below a threshold).
• Decision Making with External Data: The model can fetch external data before formulating a response (e.g., checking flight availability before booking).
• Real-Time Adaptation: Agents can adjust responses dynamically based on real-world data.

Example: Multi-Step Execution

If a user asks:

"Schedule a meeting with John about the project and check the weather in New York for that day."

The model might return:

[
  {
    "name": "get_weather",
    "arguments": {
      "location": "New York, NY"
    }
  },
  {
    "name": "create_meeting",
    "arguments": {
      "title": "Project Discussion with John",
      "date_time": "2025-02-15T10:00:00Z",
      "attendees": ["john.doe@example.com"]
    }
  }
]        

The orchestrator:

• Fetches the weather for New York.
• Schedules the meeting.
• Returns both results to the model.

GPT-4o can then respond:

"Your meeting with John is scheduled for February 15 at 10 AM. The weather in New York that day is expected to be 55°F and sunny."

The Power of AI-Orchestrated Automation

Function calling in models like GPT-4o revolutionizes AI’s role from passive chatbot to intelligent agent. By integrating real-time data, APIs, and automation tools, these models execute complex tasks, from scheduling meetings to retrieving real-world information.

This shift is paving the way for a new era of AI, where models don’t just provide information—they take action. As we continue to innovate, function calling and structured outputs will be at the core of the next generation of AI-driven automation.

While function calling and structured outputs are critical components of modern AI agents, they are just one piece of a much larger system. In future posts, we'll explore additional components of agentic AI, such as how OpenAI Assistants and Azure Agents manage stateful interactions through threads, messages, and runs.

An agent isn't just about executing individual function calls—it requires a structured process for handling context over multiple interactions, dynamically choosing when to invoke tools, and seamlessly integrating user input with system responses.

Function calling plays a key role in this ecosystem, allowing agents to execute real-world actions while maintaining continuity across interactions. Stay tuned as we dive deeper into the mechanics of full agent orchestration in upcoming discussions.