Serverless AI Function Calling and Chatbots with Cloudflare Workers AI and Llama 3.3/Qwen

The era of basic chatbots that merely generate static conversational text is over. Modern AI applications rely on Function Calling (Tool Calling) to execute real-world tasks—such as invoking external APIs, querying D1 databases, or fetching real-time weather and stock data—transforming chatbots into Autonomous Agents.

Previously, implementing Function Calling meant relying exclusively on proprietary cloud APIs (like OpenAI) or managing dedicated GPU clusters. However, Cloudflare Workers AI now directly serves state-of-the-art open models like Llama 3.3 70B (@cf/meta/llama-3.3-70b-instruct-fp8-fast) and Qwen 2.5 72B (@cf/qwen/qwen2.5-72b-instruct) across its global edge network, featuring native Function Calling support.

This article covers defining JSON Schema tools, building a complete Agent Loop to feedback tool outputs to the LLM, validating structured JSON outputs with Zod, and optimizing serverless AI costs.

Key Takeaways

• Serverless Edge AI Inference: Cloudflare Workers AI runs Llama 3.3 and Qwen 2.5 models at edge PoPs via a simple env.AI.run() call without managing GPU hardware.
• Native Function Calling Support: Pass a tools parameter containing JSON Schema specifications, and the LLM natively formats its response with the selected function name and parsed JSON arguments.
• Agent Loop Architecture: User Prompt -> LLM Analysis -> tool_calls Detection -> Worker Function Execution (D1 / external API) -> Feedback Result to Message History -> Final Answer Generation.
• Cost Advantage: At $0.011 per 1,000 Neurons alongside a free tier of 10,000 Neurons daily, Workers AI cuts AI inference costs by up to 80% compared to cloud APIs.

Architecture: Function Calling Agent Loop

[User Prompt] -> [Cloudflare Worker]
                      |
           (1) env.AI.run(tools)
                      v
            [Workers AI (Llama 3.3 / Qwen)]
                      |
       (2) Returns { tool_calls: [...] }
                      v
             [Cloudflare Worker]
                      |
    (3) Executes Tool (D1 Query / External API)
                      v
           (4) env.AI.run(messages + tool_result)
                      v
            [Workers AI Final Answer] -> [User Response]

Model Comparison for Tool Use

Complete TypeScript Implementation

export interface Env {
  AI: Ai;
  DB: D1Database;
}

const tools = [
  {
    name: "get_weather",
    description: "Fetch current weather for a specified location.",
    parameters: {
      type: "object",
      properties: {
        location: { type: "string", description: "City name e.g. Tokyo" },
        unit: { type: "string", enum: ["celsius", "fahrenheit"] },
      },
      required: ["location"],
    },
  },
  {
    name: "check_product_stock",
    description: "Query product inventory stock quantity from D1 database by SKU.",
    parameters: {
      type: "object",
      properties: {
        sku: { type: "string", description: "Product SKU code e.g. PROD-1029" },
      },
      required: ["sku"],
    },
  },
];

export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    if (request.method !== "POST") return new Response("POST only", { status: 405 });

    const { prompt } = await request.json<{ prompt: string }>();

    const messages: Array<any> = [
      { role: "system", content: "You are a helpful assistant. Use tools when necessary." },
      { role: "user", content: prompt },
    ];

    // First call to Workers AI with tools
    let response = await env.AI.run(
      "@cf/meta/llama-3.3-70b-instruct-fp8-fast",
      { messages, tools }
    );

    if (response.tool_calls && response.tool_calls.length > 0) {
      for (const toolCall of response.tool_calls) {
        const toolName = toolCall.name;
        const toolArgs = toolCall.arguments;
        let toolResult: any;

        if (toolName === "get_weather") {
          toolResult = { location: toolArgs.location, temperature: 22.5, condition: "Sunny" };
        } else if (toolName === "check_product_stock") {
          const queryResult = await env.DB.prepare(
            "SELECT sku, name, stock_quantity FROM products WHERE sku = ?"
          ).bind(toolArgs.sku).first();
          toolResult = queryResult || { error: "SKU not found" };
        }

        messages.push({ role: "assistant", tool_calls: [toolCall] });
        messages.push({ role: "tool", name: toolName, content: JSON.stringify(toolResult) });
      }

      // Second call with tool execution results added to context
      response = await env.AI.run(
        "@cf/meta/llama-3.3-70b-instruct-fp8-fast",
        { messages }
      );
    }

    return Response.json({ answer: response.response, executedMessages: messages });
  },
};

Structured JSON Output with Zod Validation

import { z } from "zod";

const ProfileSchema = z.object({
  name: z.string(),
  age: z.number(),
  skills: z.array(z.string()),
});

const res = await env.AI.run("@cf/qwen/qwen2.5-72b-instruct", {
  messages: [{ role: "user", content: "Extract profile: John, 30, skills: JS, Cloudflare" }],
  response_format: { type: "json_object" }
});

const profile = ProfileSchema.parse(JSON.parse(res.response));

Conclusion

Combining Cloudflare Workers AI with Llama 3.3 and Qwen 2.5 enables developers to deploy serverless, edge-native AI agents with native function calling capabilities at a fraction of traditional cloud costs.