Gemini API rolls out tool retrieval for efficient agent workflows

A transparent, layered gear mechanism suspended in space, where only the active gears light up and engage, while the rest remain dim and disengaged, symbolising on-demand tool activation in a large sy

Google has launched Tool Retrieval in the Gemini API, now available in preview via the Interactions API. The feature addresses inefficiencies in AI agent systems that interface with dozens or hundreds of tools by loading function schemas only when needed, reducing input token use, latency, and improving tool selection accuracy. This change is particularly relevant for developers building complex automation systems in the UK, UAE, US, and Kazakhstan, where cost and performance efficiency are critical.

How tool retrieval reduces overhead

When an AI agent is connected to a large set of tools, sending full parameter schemas with every request inflates token usage and slows response times. Tool Retrieval allows developers to defer loading of tool schemas by setting defer_loading: true. The model retains awareness of all tools through their names and descriptions but only loads the full schema when a specific tool is required. This on-demand loading happens within a single API call: the model identifies the needed tool, the API fetches its schema, and the function is executed—all visible in the interaction.steps log for debugging.

This approach avoids the need for external indexing or hosting. Google states that the only code changes required are adding a tool_search tool and flagging less frequently used tools for deferred loading. Frequently used functions can remain pre-loaded to balance speed and efficiency. The feature also complements prefix caching, as deferred schemas do not disrupt the initial prompt structure, maintaining cache consistency across turns.

Support for MCP and client-side control

Tool Retrieval integrates with remote Model Context Protocol (MCP) servers. By applying defer_loading: true to an mcp_server tool, all tools exposed by that server are loaded on demand, eliminating the need to declare them individually. This is particularly useful for applications like the Blender AI add-on mcp-for-blender-pro, which exposes 21 scene tools and supports connections to Gemini, OpenAI, and Ollama. While that add-on runs locally and manages its own MCP server, Gemini’s new capability allows cloud-based agents to scale tool access without bloating context.

For teams requiring custom control, client-side retrieval is supported. Developers can implement their own tool registry or permission filters by defining a tool_search function with execution: "client". This adds one round-trip per search but enables dynamic tool exposure during a conversation, such as when a user connects a new integration mid-session.

What it means for businesses

For UK and international tech teams, this update lowers the operational cost of deploying AI agents with broad tool access. Organisations in regulated sectors or those managing local AI models—such as in Kazakhstan’s growing 3D design and animation studios—can benefit from tighter control over data flow and reduced cloud spend. The Blender add-on example shows how such tools can integrate into creative workflows without vendor lock-in, supporting Ollama and custom endpoints.

However, companies should audit their tool usage patterns to identify which functions can be deferred. Those using MCP-based systems should evaluate whether Gemini’s server-side retrieval meets their security and compliance needs, especially if client-side filtering is required for access control.

Organisations using GitHub Copilot can now opt for Gemini 4 Neon across IDEs including VS Code, JetBrains, and Xcode. Administrators can manage access via model policies, with gradual rollout in progress. This model’s efficiency gains make it a strong candidate for enterprise coding automation where cost and accuracy are prioritised.

Sources

Leave a Reply

Your email address will not be published. Required fields are marked *