Overview
An AI agent is an LLM that can take actions — call tools (APIs, databases, code executors), observe results, and decide the next step. Unlike a single LLM call (question → answer), an agent runs a loop: Think → Act → Observe → Think → Act... until it reaches a goal or hits a stop condition.
Production agents need:
- Tool registry — defined functions the model can call
- Execution engine — runs tool calls safely (sandboxed, rate-limited)
- State management — persists conversation + tool call history
- Guardrails — max iterations, timeout, cost limit, human escalation
- Observability — trace every reasoning step for debugging
Architecture
┌──────────────────────────────────────────────────────────────┐
│ Agent Orchestration Layer │
│ │
│ User Request / Event Trigger │
│ │ │
│ ▼ │
│ ┌─────────────┐ ┌────────────────────────────────┐ │
│ │ Agent │ │ Tool Registry │ │
│ │ Controller │ │ │ │
│ │ │ │ search_knowledge_base() │ │
│ │ - max 10 │ │ create_jira_ticket() │ │
│ │ iterations│ │ query_database(sql) │ │
│ │ - $0.50 │ │ send_email(to, subject, body) │ │
│ │ cost cap │ │ call_api(method, url, body) │ │
│ │ - 60s │ │ run_code(language, code) │ │
│ │ timeout │ └────────────┬───────────────────┘ │
│ └──────┬──────┘ │ tool definitions │
│ │ │ (JSON schema) │
│ ▼ ▼ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ LLM (Claude / GPT-4o) │ │
│ │ Input: conversation history + tool definitions │ │
│ │ Output: text OR tool_call{name, args} │ │
│ └───────────────────────┬─────────────────────────────┘ │
│ │ │
│ ┌──────────────▼──────────────┐ │
│ │ Tool call? │ │
│ │ YES ──▶ Execute Tool │ │
│ │ │ │ │
│ │ ▼ │ │
│ │ Append result │ │
│ │ to history │ │
│ │ │ │ │
│ │ └──▶ Loop back │ │
│ │ NO ──▶ Final Answer │ │
│ └────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘
━━━━━━━━━ EVENT-DRIVEN AI PIPELINE ━━━━━━━━━
Kafka Topic Agent Service Downstream
┌────────────┐ ┌──────────────┐ ┌──────────┐
│ user.events│──────────────▶│ │───────▶│ CRM API │
│ (SQS/Kafka)│ │ LLM Agent │ └──────────┘
└────────────┘ │ │ ┌──────────┐
│ - classify │───────▶│ Slack │
┌────────────┐ │ - enrich │ └──────────┘
│ order.fail │──────────────▶│ - route │ ┌──────────┐
└────────────┘ │ - respond │───────▶│ Jira │
└──────────────┘ └──────────┘
Technical Implementation
Tool Definition and Registration (TypeScript)
// src/agents/tools/index.ts
import { Tool } from '@anthropic-ai/sdk';
export const tools: Tool[] = [
{
name: 'search_knowledge_base',
description: 'Search the internal knowledge base for relevant documents. Use this when the user asks a question that requires internal documentation.',
input_schema: {
type: 'object',
properties: {
query: { type: 'string', description: 'The search query' },
topK: { type: 'number', description: 'Number of results to return (default 5)', default: 5 },
},
required: ['query'],
},
},
{
name: 'create_jira_ticket',
description: 'Create a Jira ticket for a bug or feature request.',
input_schema: {
type: 'object',
properties: {
title: { type: 'string' },
description: { type: 'string' },
priority: { type: 'string', enum: ['low', 'medium', 'high', 'critical'] },
assignee: { type: 'string', description: 'Jira username' },
},
required: ['title', 'description', 'priority'],
},
},
{
name: 'query_database',
description: 'Run a read-only SQL query against the analytics database. Only SELECT statements allowed.',
input_schema: {
type: 'object',
properties: {
sql: { type: 'string', description: 'SQL SELECT statement' },
},
required: ['sql'],
},
},
];
Agent Execution Loop
// src/agents/agent.service.ts
import Anthropic from '@anthropic-ai/sdk';
export class AgentService {
private client = new Anthropic();
private readonly MAX_ITERATIONS = 10;
private readonly MAX_COST_USD = 0.50;
async run(userMessage: string, context: AgentContext): Promise<AgentResult> {
const messages: Anthropic.MessageParam[] = [
{ role: 'user', content: userMessage }
];
let iterations = 0;
let totalCostUsd = 0;
while (iterations < this.MAX_ITERATIONS) {
iterations++;
const response = await this.client.messages.create({
model: 'claude-opus-4-7',
max_tokens: 4096,
system: this.buildSystemPrompt(context),
tools,
messages,
});
// Track cost
totalCostUsd += this.calculateCost(response.usage);
if (totalCostUsd > this.MAX_COST_USD) {
return this.buildResult(messages, 'COST_LIMIT_EXCEEDED');
}
// End of reasoning — final answer
if (response.stop_reason === 'end_turn') {
const textBlock = response.content.find((b) => b.type === 'text');
return this.buildResult(messages, 'SUCCESS', textBlock?.text);
}
// Model wants to call tools
if (response.stop_reason === 'tool_use') {
messages.push({ role: 'assistant', content: response.content });
// Execute all tool calls in parallel
const toolResults = await Promise.all(
response.content
.filter((b): b is Anthropic.ToolUseBlock => b.type === 'tool_use')
.map((toolUse) => this.executeTool(toolUse, context))
);
messages.push({ role: 'user', content: toolResults });
}
}
return this.buildResult(messages, 'MAX_ITERATIONS_EXCEEDED');
}
private async executeTool(
toolUse: Anthropic.ToolUseBlock,
context: AgentContext
): Promise<Anthropic.ToolResultBlockParam> {
try {
let result: unknown;
switch (toolUse.name) {
case 'search_knowledge_base':
result = await this.ragService.query(
(toolUse.input as { query: string }).query, context.tenantId
);
break;
case 'create_jira_ticket':
result = await this.jiraService.create(toolUse.input as JiraTicketInput);
break;
case 'query_database':
// Validate SQL is read-only before execution
const sql = (toolUse.input as { sql: string }).sql;
if (!/^SELECT/i.test(sql.trim())) {
throw new Error('Only SELECT queries are allowed');
}
result = await this.analyticsDb.query(sql);
break;
default:
throw new Error(`Unknown tool: ${toolUse.name}`);
}
return {
type: 'tool_result',
tool_use_id: toolUse.id,
content: JSON.stringify(result),
};
} catch (err) {
return {
type: 'tool_result',
tool_use_id: toolUse.id,
is_error: true,
content: `Tool error: ${(err as Error).message}`,
};
}
}
}
Human-in-the-Loop Checkpoint
// src/agents/hitl.service.ts
// For high-stakes actions, pause and get human approval before executing
export class HITLService {
private pendingApprovals = new Map<string, PendingApproval>();
async requireApproval(
action: ToolCallAction,
context: AgentContext
): Promise<'approved' | 'rejected'> {
const approvalId = crypto.randomUUID();
// Notify human reviewer (Slack, email, etc.)
await this.notifier.sendApprovalRequest({
id: approvalId,
action: action.name,
args: action.args,
summary: action.humanReadableSummary,
approveUrl: `${process.env.APP_URL}/approvals/${approvalId}/approve`,
rejectUrl: `${process.env.APP_URL}/approvals/${approvalId}/reject`,
expiresAt: new Date(Date.now() + 30 * 60 * 1000), // 30 min timeout
});
// Poll for decision (store approval state in Redis/DB)
return this.waitForDecision(approvalId, 30 * 60 * 1000);
}
// Actions that always require human approval
static requiresApproval(toolName: string): boolean {
return ['send_email', 'create_payment', 'delete_record', 'deploy_code'].includes(toolName);
}
}
Event-Driven Agent Pipeline (Kafka)
// src/agents/event-agent.consumer.ts
// Agent triggered by Kafka events — processes async, no user waiting
@Injectable()
export class EventAgentConsumer {
@KafkaEventHandler('user.support.request')
async handleSupportRequest(event: SupportRequestEvent): Promise<void> {
const result = await this.agentService.run(
`Classify and respond to this support request: "${event.message}"
User tier: ${event.userTier}. Order ID: ${event.orderId ?? 'none'}`,
{ tenantId: event.tenantId, userId: event.userId }
);
if (result.status === 'SUCCESS') {
// Route based on agent's classification
const classification = JSON.parse(result.finalAnswer);
if (classification.requiresHuman) {
await this.crmService.createTicket(event, classification);
} else {
await this.emailService.sendResponse(event.userId, classification.response);
}
await this.producer.emit('support.request.resolved', {
requestId: event.id,
resolution: classification.category,
agentIterations: result.iterations,
});
}
}
}
Interview Preparation
Q: What's the difference between a chain and an agent?
A chain is a fixed sequence of LLM calls (prompt A → output → prompt B). An agent has a dynamic loop — the model decides at each step whether to call a tool or produce a final answer. Chains are predictable and testable; agents are flexible but harder to control. Use chains when the steps are known; agents when the steps depend on intermediate results.
Q: How do you prevent an agent from looping forever?
Hard limits enforced outside the LLM: max iteration count (10–20), wall-clock timeout (60s), cost cap (e.g., $0.50 per run). Also: detect repeated tool calls with identical args (the agent is stuck), and break with an error.
Q: How do you test agent behavior?
- Unit test tool executors independently — mock the LLM, verify tool logic
- Snapshot test conversation traces — given this history, assert the agent calls the right tool
- Eval dataset — 50–100 representative user inputs with expected outcomes; measure success rate
- Red-team — explicitly try to get the agent to call dangerous tools or ignore its system prompt
Q: When does a tool call need human approval?
Any action that is irreversible or high-consequence: sending emails to customers, creating charges, modifying production data, deploying code. The pattern: before executing the tool, emit an approval request event and pause the agent's execution coroutine until approved (or timed out → default to reject).
Q: How do you handle tool failures in an agent loop?
Return the error as a tool_result with is_error: true. The model is trained to handle errors and will typically either retry with different args, try a different tool, or tell the user it couldn't complete the task. Don't throw exceptions that kill the loop — graceful degradation matters.
Learning Resources
- Anthropic Tool Use Guide — official tool calling docs
- OpenAI Function Calling
- LangGraph — stateful agent graph framework
- Building Effective Agents — Anthropic's guide
- ReAct Paper — Reasoning + Acting, the foundation of most agent loops
- Gorilla LLM — LLM tool use research