Understanding Agentic Function-Calling with Multi-Modal Data Access

This guide explains how to build agentic systems using function-calling in LLMs to access and combine data from multiple sources. Function-calling allows LLMs to use tools, such as SQL queries or file readers, instead of providing text-only answers. Agentic systems are characterized by autonomy, tool use, and iterative reasoning, enabling multi-step problem-solving. The iterative tool-use loop facilitates this reasoning by allowing the agent to call tools, analyze results, and repeat as needed. Multi-modal data access combines the strengths of various data stores like SQL databases and blob storage. The cross-reference pattern, using a BlobPath column, is a key architectural element. Effective system prompts, including schema injection and dialect rules, are essential for agent behavior. Tool design is crucial, focusing on single responsibilities, rich descriptions, structured data returns, graceful failure handling, and limited scopes. The LLM's decision-making process considers tool descriptions, system prompts, results, and conversation history. Agents maintain context through conversation memory, incorporating past turns in subsequent interactions.