OpenAI's New Built-in Tools: Towards Agentic Systems

OpenAI has recently introduced a suite of built-in tools that extend the capabilities of its AI models, making it easier for developers to build powerful AI-driven applications. These tools are part of OpenAI’s evolving platform (notably the new Responses API and Agents SDK) geared towards creating agentic AI systems – AI agents that can independently perform tasks on behalf of users. In this article, we provide an overview of these new tools, explain their purpose and how they enhance AI applications, explore real-world use cases, and demonstrate with code examples how to use them. We’ll also cover best practices to implement these tools effectively in your own projects.

Overview of OpenAI's Built-in Tools and Their Purpose

OpenAI’s built-in tools allow AI models to access external information or perform actions, overcoming some limitations of standalone LLMs. The current set of new tools includes:

• Web Search – Allows the AI to query the internet in real-time and retrieve up-to-date information, returning answers with clear source citations. This helps models answer questions about current events or facts beyond their training data.
• File Search – Enables the AI to search through large volumes of documents or knowledge bases provided by the developer. It uses a vector database of files to find relevant information, allowing the model to answer queries using custom or private data (ideal for retrieval-augmented generation).
• Computer Use – Lets the AI simulate a user operating a computer (e.g. performing mouse clicks, typing, browsing) to accomplish tasks in a software or web environment. This effectively allows an AI agent to interact with applications or websites and automate workflows that normally require human computer interactions.

Each tool serves a specific purpose: Web Search brings in fresh knowledge and transparency (via citations), File Search injects domain-specific or proprietary data into the AI’s context, and Computer Use grants the AI the ability to take actions in digital environments. Together, these tools greatly enhance what AI applications can do, enabling more useful and reliable agents that can retrieve information and take actions autonomously.

Web Search Tool – Real-time Information with Citations

The Web Search tool gives AI models access to the vast information on the internet. Instead of relying solely on the model’s pre-trained knowledge, the AI can perform live web queries and incorporate the results into its responses. This means the model can fetch fast, up-to-date answers from the web, complete with relevant source links. The answers generated through this tool include inline citations (e.g. links to news articles or websites), allowing users to verify facts and explore sources further. This transparency builds trust and enables new user experiences where the AI not only provides information but also shows where that information came from.

How it enhances applications: With web search, AI applications are no longer limited by a static knowledge cutoff. They can handle queries about recent news, live sports scores, stock prices, or any timely data. For example, an AI assistant could answer “What happened in tech news this morning?” by searching the web and giving an answer with references to news sites. This opens the door to applications like real-time research assistants, shopping advisors, or travel planners that always use the latest information. During early testing, developers have built shopping assistants, research agents, and travel booking agents – essentially any use-case requiring timely web info can benefit from this tool.

Real-world example: One company leveraging web search is Hebbia. Hebbia integrates OpenAI’s web search tool to help financial and legal professionals quickly extract insights from extensive public and private data sources. By incorporating real-time web search into their workflow, Hebbia’s AI agent can deliver richer, context-specific market intelligence and consistently improve the relevance of its analyses, outperforming previous benchmarks. In essence, web search empowers such agents to be significantly more knowledgeable and up-to-date.

Using the Web Search tool – step-by-step example: OpenAI’s API provides an easy way to use the web search tool in your requests. Under the hood, this tool is available when using the gpt-4o or gpt-4o-mini model in the new Responses API (these are fine-tuned models optimized for tool use). Here’s how you can use the web search tool in practice:

1. Set up the API request – You’ll need to call the Responses API endpoint and include the web search tool in the request payload. In code, this means specifying a tools parameter with the type "web_search_preview" (the preview version of the web search tool). You also choose a model that supports this tool (e.g. "gpt-4o").
2. Provide the user query – You can then send the user’s question or prompt as the input. The model will decide if and how to use the web search tool to get additional information before responding.
3. Process the response – The API will return a response that includes the model’s answer in output_text, typically enriched with the information found online and citations to the sources.

For example, using the OpenAI Python library (or an equivalent HTTP request), a call might look like this:

import openai
openai.api_key = "YOUR_API_KEY"

# Ask a question with the web search tool enabled
response = openai.Responses.create(
    model="gpt-4o",
    tools=[{"type": "web_search_preview"}],
    input="What was a positive news story that happened today?"
)
print(response.output_text)
        

In this snippet, we request GPT-4 to find a positive news story from today. The model will perform a web search under the hood and then produce an answer. The printed output_text might be a summary of the news story along with a citation link to the source. For example, it could return something like:

"One positive news story today is that scientists announced a breakthrough in renewable energy storage, which could accelerate the adoption of clean energy."

The citation in the text (shown here as a reference) would point to a specific article or source supporting the statement. By including citations, the web search tool not only gives the latest information but also enhances trust and verifiability in AI-generated content.

File Search Tool – Leveraging Custom Data and Documents

The File Search tool allows your AI assistant to ingest and retrieve information from your own documents or knowledge base. This is extremely useful for enterprise and domain-specific applications, where answers need to be based on proprietary data (such as product documentation, internal wikis, PDFs of research papers, etc.). OpenAI’s file search works by indexing documents in a vector store and then querying that store for relevant content when the model receives a prompt. Under the hood, it uses embeddings and similarity search to find text segments that are relevant to the query, and makes those available to the model.

How it enhances applications: By using file search, developers can build retrieval-augmented generation (RAG) pipelines with very little effort. The model can pull in knowledge from large volumes of text that it otherwise wouldn’t know. This means an AI app can have up-to-date company policies, product specs, manuals, or any corpus of choice at its fingertips. The file search tool supports multiple file types and offers features like query optimization, metadata filtering, and result reranking for accuracy (New tools for building agents | OpenAI). Essentially, it brings enterprise knowledge integration to AI: a customer support bot can fetch answers from FAQ documents, a legal assistant bot can cite relevant case files, or a coding helper can look up an API reference – all through the same unified model interface.

Use cases and benefits: There are many real-world scenarios for file search. For instance, a customer support agent AI could instantly retrieve answers from a company’s FAQ or knowledge base, providing customers with precise information. A legal research assistant AI might quickly scan a database of court cases or regulations to find relevant passages for a lawyer’s query. A developer’s coding assistant could search through documentation or a codebase to help answer technical questions. In fact, the tool has been used to enable things like a travel agent AI retrieving company travel policy details for users. One example is Navan, which uses the file search tool in its AI travel assistant to provide users with precise answers from internal knowledge-base articles, such as a company’s travel policy. Thanks to built-in query tuning and reranking, Navan set up a powerful RAG system without extra custom code or tuning. By maintaining dedicated vector stores for different user groups, they even tailor answers to each customer’s context, resulting in more accurate and personalized support. This demonstrates how file search can save time and improve accuracy for end-users by drawing on the right piece of information at the right moment.

Using the File Search tool – step-by-step example: To use file search, you need to first prepare your document data and then make a query with the tool. Here’s a typical workflow:

1. Upload documents and create a vector store – You upload your files via OpenAI’s API or web interface and create a VectorStore (an index of these files). Each file is processed into embeddings for efficient search. In code, this might be done with a call like openai.vectorStores.create(...) by providing file IDs. (This step is done once, whenever you need to index new documents.)
2. Query using the file search tool – When making a request to the model, include the file_search tool and specify which vector store(s) the model should search. You also provide the user’s query as input.
3. Get the answer with referenced info – The model will search the specified document store for relevant content, incorporate what it finds into its reasoning, and return a response that likely includes details from your documents (and possibly citations or references to them, if you’ve enabled that).

For example, here’s how you might implement a file search query in Python:

# (Assume openai.api_key is already set and you have file IDs from previously uploaded files)
# 1. Create a vector store for your documents
product_docs = openai.VectorStore.create(
    name="Product Documentation",
    file_ids=[file1_id, file2_id, file3_id]  # IDs of documents to index
)

# 2. Use the file_search tool in a model request
response = openai.Responses.create(
    model="gpt-4o-mini",
    tools=[{
        "type": "file_search",
        "vector_store_ids": [product_docs.id]  # reference the created vector store by ID
    }],
    input="What is 'Deep Research' in the context of OpenAI?"  # sample user query
)

# 3. Output the result
print(response.output_text)
        

In this code, we first create a vector store named "Product Documentation" containing three files (perhaps PDFs or text files of product research). Then we ask the model a question: “What is 'Deep Research' by OpenAI?” including the file search tool with our product_docs store. The model will search those documents for the term "Deep Research" and related context. The output_text we get back might be something like:

“‘Deep Research’ is an OpenAI initiative aimed at long-term fundamental research in AI, focusing on high-risk, high-reward ideas. According to OpenAI’s documentation, Deep Research involves collaborations with academic institutions and exploration of new learning paradigms.”

If that information was contained in one of the provided documents, the model can extract and present it. In practice, the answer might also cite the document title or source. By using file search, we ensure the assistant’s answer is grounded in the actual data we provided, which greatly improves accuracy and usefulness in domain-specific applications.

Computer Use Tool – Automating Actions in an Environment

The Computer Use tool is perhaps the most groundbreaking of the new built-in tools. It gives the AI the ability to perform actions on a computer — essentially controlling a virtual computer interface (like moving a mouse, clicking, typing, scrolling). This is powered by OpenAI’s Computer-Using Agent (CUA) model, a specialized model that can interpret instructions and carry out sequences of GUI operations. In the OpenAI API, the computer use tool captures the model’s suggested mouse and keyboard actions and translates them into executable commands in a controlled environment (New tools for building agents | OpenAI). In simpler terms, the AI can operate software or websites for you, under supervision.

How it enhances applications: This capability turns an AI model into an agent that can not only think or generate text, but also act. It can handle tasks that typically require a human using a computer. For example, an AI agent could fill out forms on a website, navigate through a web app, or cross-reference information by clicking through pages. This is extremely useful for automating workflows where no convenient API exists. Developers can use it to perform browser-based tasks like web app testing, data entry in legacy systems, or any repetitive online operation. Essentially, it’s like an AI-powered Robotic Process Automation (RPA) but driven by natural language instructions.

Use cases and real-world examples: The computer use tool can be applied to a wide range of scenarios:

• Automated web testing and QA: You can instruct an agent to go through a series of user actions on a web application to ensure everything works as expected.
• Data entry and migration: If a company has an old system without APIs, an AI agent could input data or transfer information between systems by literally using the UI.
• Research and verification tasks: As an example, Unify (a go-to-market automation company) uses OpenAI’s computer use tool to have agents gather information not accessible via APIs. In one case, an agent checks an online mapping service to see if a business has expanded its locations – a task which provides valuable sales intelligence. The AI essentially “looks at” the map website like a human would, and then uses that insight to trigger personalized outreach.
• Complex workflow automation: Luminai is another company that integrated the computer use tool, using it to automate complicated operational workflows for enterprises with legacy software. In a pilot, Luminai’s AI agent managed to complete an end-to-end user enrollment process by navigating a legacy system’s interface – accomplishing in days what had stymied traditional RPA solutions for months. This highlights how the tool can handle complex, multi-step tasks by intelligently controlling a UI, offering huge productivity gains in settings where direct integration isn’t possible.

By enabling these use cases, the computer use tool effectively allows AI to bridge the gap between language understanding and real-world action. It enhances AI applications by letting them interact with the same interfaces humans use, which is a significant step toward more autonomous agents.

Using the Computer Use tool – step-by-step example: The computer use tool is currently available as a research preview (to select developers in higher usage tiers), which means you may need special access to experiment with it. Assuming you have access, here’s how you can use it:

1. Set up the request with the special model and tool – The computer control is enabled by using the model "computer-use-preview" (a model designed for this capability) and including the "computer_use_preview" tool in your request. You also need to specify some parameters for the virtual environment, such as the display width/height (simulating screen size) and the type of environment (e.g. "browser" for a web browser interface).
2. Provide a high-level instruction – You then give the model a prompt describing the task. For instance, you might say, “Find me the best price of a particular camera on the web”, which implies the agent should search online stores via a browser.
3. Process the action output – The response from the API will include the actions the AI took or the final result of those actions. The output may contain structured data about the clicks/keystrokes performed, or the end state (like text the agent found or entered). Your application can then interpret or validate these results (e.g., check that the desired outcome was achieved).

Here is an illustrative example using the OpenAI API (pseudo-code in Python format):

# Using the computer use tool to have the AI perform a task in a browser.
response = openai.Responses.create(
    model="computer-use-preview",
    tools=[{
        "type": "computer_use_preview",
        "display_width": 1024,
        "display_height": 768,
        "environment": "browser"
    }],
    input="I'm looking for a new camera. Help me find the best one."
)
print(response.output)
        

In this example, the AI is instructed to help find the best camera. What happens behind the scenes is the model will likely open a browser (in the sandboxed environment), perform a search for cameras, navigate through search results or shopping sites, and compile the information. The response.output could be a structured log or description of what it did (for instance, “Opened example.com, searched for ‘best camera 2025’, clicked the first result, price noted was $X”). It might also return a synthesized answer like, “The XYZ Camera is highly rated this year for its image quality and is available for $799 on CameraShop.com.” along with details of how it found that info.

Safety and control: Because this tool actually carries out actions, OpenAI has built in multiple safeguards. The model’s actions are executed in a controlled environment (not on your actual machine). OpenAI’s research showed the CUA model achieves impressive results on benchmarks for computer tasks (e.g., setting a new state-of-the-art on the WebArena browsing benchmark), but it’s not perfect and can make mistakes. Thus, the platform has safety checks and confirmations in place. For instance, the system may require the AI to get user confirmation before performing sensitive operations (like deleting data or making a purchase), and there are filters to guard against malicious prompt injections that might try to exploit the tool (Openai Agents SDK, Responses Api Tutorial - DEV Community). As a developer, you should also implement checks on your side – for example, reviewing the actions an agent wants to take, and sandboxing the environment (which OpenAI enables) so no unintended harm can occur. We’ll discuss more best practices on this below.

Best Practices for Implementing Built-in Tools Effectively

Using OpenAI’s built-in tools can greatly enhance your AI application, but to get the best results and ensure safety, keep these best practices in mind:

• Select the right tool (or tools) for the task: Choose the tool that fits the user’s query or desired action. For instance, use Web Search for questions about recent or external information, File Search for queries answerable from your custom data, and Computer Use for tasks that require interacting with a user interface. The Responses API allows you to enable multiple tools at once or even combine tool use with function calling for complex tasks. Design your agent’s prompt or system instructions to clarify which tool is available so the model knows it can use them.
• Provide clear instructions and context: Even with tools, the AI’s behavior is guided by its prompt. Set the scene for the model – you might use a system message like “You are an assistant with access to web search and file search tools. Use them when necessary to find accurate information.” This primes the model to take advantage of the tools. Be explicit if certain information must come from a specific source (e.g., “The answer should be based on our internal documents.”). Clear instructions will lead to the model invoking tools more effectively and only when appropriate.
• Monitor and validate the outputs: Always inspect the AI’s outputs, especially when tools are used. With Web Search, check the citations or references the model provides – ensure they actually support the answer and that the links are relevant (the model might sometimes misquote or misattribute if it’s not accurate, so verification is key). For Computer Use, you should log or review the sequence of actions the AI took (which the API can return) to confirm it performed the steps correctly. Essentially, keep a human-in-the-loop for critical tasks. OpenAI’s own testing emphasizes that human oversight is recommended for computer-use actions since the model can be error-prone in complex scenarios.
• Implement guardrails for safety: With great power comes great responsibility – an AI that can search the web or operate a computer should be fenced in by safety checks. OpenAI has implemented several safety mitigations for these tools (for example, checks against prompt injection, confirmation prompts for risky actions, environment isolation, and policy violation detection). As a developer, you should use these features and add your own where possible. For example, you might require user confirmation before an AI agent makes a purchase or sends an email on the user’s behalf. If the AI is using web search, consider filtering or restricting which sites it can access if that’s a concern. Leverage OpenAI’s guardrail configurations (like the guardrail parameter in the Agents SDK) or build custom validation logic to intercept any potentially harmful or unwanted actions.
• Test iteratively in a safe environment: Before deploying an agent broadly, test its tool-using behavior in a controlled setting. Use OpenAI’s Playground or sandbox environment to simulate queries and see how the model uses the tools. This will help you fine-tune prompts and catch odd behaviors. For the computer use tool, testing in a sandbox (with a dummy account or test data) is crucial to observe what the AI does. Start with read-only or non-destructive tasks to build trust in the agent’s capabilities. As you gain confidence, you can scale up to more complex or sensitive tasks gradually.
• Leverage observability and logs: OpenAI’s platform provides observability tools to trace and inspect agent workflows. Take advantage of these to debug the chain of actions and tool calls. When your AI agent uses a tool, the API can return intermediate information (like what query was sent to web search or which document was retrieved by file search). Logging these details will help you understand the AI’s decision process and improve it. For instance, if the AI isn’t finding the right info from file search, you might need to enrich your documents or adjust the query. Observability helps in optimizing performance and ensuring reliability.
• Maintain and update your knowledge base: If you are using File Search, remember that the answers are only as good as the data provided. Keep your vector stores updated with the latest documents. If something changes (like policies, product specs, etc.), re-index those files so the AI has current information. Similarly for Web Search, if you have a specific set of sites that are authoritative for your domain, you might incorporate that into the prompt (e.g., instruct the AI to prioritize certain sources) or use OpenAI’s feature for allowing/disallowing certain domains.
• Be mindful of costs and performance: Each tool usage may have associated costs (OpenAI’s tools are priced per query or token usage as noted in their documentation). Web searches and file searches will add some latency to the response as they involve external calls or lookups. Use tools when they add clear value, and consider strategies like caching results for frequent queries or limiting the scope of search when possible (for example, if you only need to search a specific site or a subset of documents, specify that). Best practice is to balance accuracy with efficiency – use the tool to the extent needed to get a correct answer, but not unnecessarily.

By following these practices, you can harness OpenAI’s built-in tools to build more powerful, safe, and reliable AI applications. Whether you’re creating an AI assistant that surfs the web for answers, a chatbot that consults enterprise knowledge bases, or an autonomous agent that navigates software interfaces, these tools provide the building blocks to do so with minimal overhead.

Conclusion

OpenAI's new built-in tools mark an exciting advancement in the AI developer toolkit. They effectively extend an AI model’s abilities beyond text generation to include searching knowledge (Web Search), leveraging custom data (File Search), and taking actions (Computer Use). This built-in functionality can dramatically improve what AI applications can do – enabling them to provide up-to-date information, grounded answers from proprietary data, and even automate tasks in the digital world. The introduction of these tools (alongside the Responses API and Agents SDK) has streamlined the core logic and orchestration required to build AI agents, making it significantly easier for developers to get started.

As we have seen, the potential use cases are vast: from enhancing customer support and personal assistants to automating business processes and beyond. By incorporating the best practices outlined above – choosing the right tool, guiding the model properly, and implementing safety checks – developers and AI enthusiasts can build innovative applications that were not previously possible with standalone GPT-style models. OpenAI continues to improve these tools (currently some are in preview) and plans to introduce even more capabilities in the future..

In summary, OpenAI’s built-in tools empower you to build AI systems that learn, look up information, and act. With careful implementation, they can unlock a new generation of AI applications that are more knowledgeable, context-aware, and action-oriented. Whether you're building a smart research agent or an autonomous helper for tedious tasks, these tools provide a powerful foundation to bring your AI ideas to life. Happy building!