Stage 7 · Master
LLM Foundations for Operators
Tool Calling & Function APIs
Letting a model run kubectl, query Prometheus, or open a PR safely.
What Is Tool Calling?
Tool calling lets an LLM request that your application execute a function. Instead of generating text, the model outputs a structured JSON request for a specific tool with specific parameters. Your application validates, executes the tool, and returns the result to the model.
This is how you give an LLM real capabilities — querying Prometheus, running kubectl, checking PagerDuty — without the model hallucinating commands or accessing systems directly.
Defining Tools
tools = [
{
"type": "function",
"function": {
"name": "query_prometheus",
"description": "Execute a PromQL query and return results",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "The PromQL query to execute"
},
"time_range": {
"type": "string",
"description": "Time range like '1h', '6h', '24h'",
"default": "1h"
}
},
"required": ["query"]
}
}
},
{
"type": "function",
"function": {
"name": "run_kubectl",
"description": "Run a read-only kubectl command",
"parameters": {
"type": "object",
"properties": {
"command": {
"type": "string",
"description": "kubectl command without the 'kubectl' prefix"
},
"namespace": {
"type": "string",
"description": "Kubernetes namespace"
}
},
"required": ["command"]
}
}
}
]Each tool has a name, description, and JSON Schema for its parameters. The description is critical — the model uses it to decide which tool to call and what arguments to pass.
The model relies on your tool descriptions to understand when and how to use each tool. Be specific about what the tool does, what inputs it expects, and what it returns.
Executing Tool Calls
import openai
import json
def execute_tool(name, arguments):
if name == "query_prometheus":
return prometheus_client.query(
arguments["query"],
time_range=arguments.get("time_range", "1h")
)
elif name == "run_kubectl":
result = subprocess.run(
["kubectl"] + arguments["command"].split(),
capture_output=True, text=True
)
return result.stdout
raise ValueError(f"Unknown tool: {name}")
def chat_with_tools(messages, tools):
response = openai.ChatCompletion.create(
model="gpt-4",
messages=messages,
tools=tools,
tool_choice="auto"
)
msg = response["choices"][0]["message"]
if msg.get("tool_calls"):
messages.append(msg)
for tool_call in msg["tool_calls"]:
result = execute_tool(
tool_call["function"]["name"],
json.loads(tool_call["function"]["arguments"])
)
messages.append({
"role": "tool",
"tool_call_id": tool_call["id"],
"content": json.dumps(result)
})
return chat_with_tools(messages, tools)
return msg["content"]The model outputs tool calls. Your application executes them and feeds results back. The model then uses those results to formulate its final answer.
Multi-Step Tool Chains
Real operational tasks often require multiple tool calls. The model might query Prometheus, then check Kubernetes, then look up a runbook. Each tool call result informs the next step.
# User asks: "Why is the checkout service slow?"
# Step 1: Model calls query_prometheus
# -> Finds p99 latency spike at 14:32
# Step 2: Model calls run_kubectl("get pods -l app=checkout")
# -> Sees pod restarts at 14:30
# Step 3: Model calls query_prometheus with OOM query
# -> Confirms OOMKills starting at 14:30
# Step 4: Model synthesizes answer:
# "The checkout service is experiencing OOMKills since 14:30,
# causing pod restarts and p99 latency spikes.
# The memory limit of 256Mi appears insufficient.
# Recommend increasing to 512Mi."Each step builds on the previous one. The model decides which tools to call based on what it has learned so far.
Safety Patterns
Tool calling gives the model real-world capabilities. This requires safety guardrails to prevent unintended consequences.
- Read-only by default — kubectl get and describe are safe; kubectl delete is not.
- Approval gates — require human confirmation for any write operation.
- Scope restrictions — limit which namespaces, clusters, or resources the model can access.
- Dry-run mode — show the command that would be executed before actually executing it.
- Rate limiting — prevent the model from making too many tool calls in a single conversation.
A single hallucinated kubectl delete command can take down production. Always require human approval for destructive operations, and use RBAC to enforce minimum permissions.
SAFE_TOOLS = ["query_prometheus", "run_kubectl_get", "search_runbook"]
REQUIRES_APPROVAL = ["run_kubectl_delete", "trigger_failover", "create_incident"]
def execute_tool_with_approval(name, arguments):
if name in REQUIRES_APPROVAL:
print(f"Tool requires approval: {name}({arguments})")
approval = input("Approve? (yes/no): ")
if approval != "yes":
return "Tool call denied by user"
return execute_tool(name, arguments)Separate safe tools from dangerous ones. Require explicit human approval for anything that could cause impact.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.