Building Real-Time Speech-to-Text (STT) Pipelines with Cloudflare Workers AI and Whisper

Building Real-Time Speech-to-Text (STT) Pipelines with Cloudflare Workers AI and Whisper

Speech-to-Text (STT) capabilities have become foundational for AI subtitling, audio note-taking, and live meeting transcription. However, operating self-hosted GPU instances for OpenAI’s Whisper model or paying high third-party API fees creates substantial cost and operational overhead.

Cloudflare Workers AI eliminates these headaches by running OpenAI’s Whisper model (@cf/openai/whisper) directly on Cloudflare’s global edge network equipped with serverless GPUs, delivering ultra-low latency without server management.

In this article, we demonstrate how to build an end-to-end real-time STT streaming pipeline by connecting browser Web Audio APIs with Cloudflare Workers AI.

1. Benefits: Traditional GPU Servers vs. Cloudflare Workers AI Whisper

2. Workers AI Whisper Integration (wrangler.jsonc & Worker Code)

Inside a Cloudflare Worker, AI bindings enable direct execution of the Whisper model without requiring external API keys.

1) wrangler.jsonc Configuration

{
  "name": "whisper-stt-worker",
  "main": "src/index.ts",
  "compatibility_date": "2026-07-01",
  "ai": {
    "binding": "AI"
  }
}

2) Worker STT Handler (src/index.ts)

export interface Env {
  AI: any;
}

export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    if (request.method !== 'POST') {
      return new Response('Method Not Allowed', { status: 405 });
    }

    try {
      // Audio binary buffer sent from the browser client
      const audioBuffer = await request.arrayBuffer();

      // Invoke Whisper model using Cloudflare Workers AI binding
      const response = await env.AI.run('@cf/openai/whisper', {
        audio: [...new Uint8Array(audioBuffer)],
      });

      // Return transcription response
      return Response.json({
        text: response.text,
        word_count: response.words?.length || 0,
        vtt: response.vtt, // WebVTT format for captions
      });
    } catch (error: any) {
      return Response.json({ error: error.message }, { status: 500 });
    }
  },
};

3. Client Web Audio Recorder and Chunked Streaming

This frontend implementation uses the browser’s MediaRecorder API to capture microphone input into 2-second audio chunks and stream them to the Worker.

<!-- index.html -->
<button id="startBtn">Start Recording</button>
<button id="stopBtn" disabled>Stop</button>
<div id="transcript"></div>

<script>
  let mediaRecorder;
  const transcriptDiv = document.getElementById('transcript');

  document.getElementById('startBtn').onclick = async () => {
    const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
    mediaRecorder = new MediaRecorder(stream, { mimeType: 'audio/webm' });

    mediaRecorder.ondataavailable = async (event) => {
      if (event.data.size > 0) {
        const audioBlob = event.data;
        const arrayBuffer = await audioBlob.arrayBuffer();

        // Send audio chunk to Cloudflare Worker endpoint
        const res = await fetch('/api/stt', {
          method: 'POST',
          headers: { 'Content-Type': 'application/octet-stream' },
          body: arrayBuffer
        });

        const result = await res.json();
        if (result.text) {
          transcriptDiv.innerText += " " + result.text;
        }
      }
    };

    // Emit audio chunks every 2000ms
    mediaRecorder.start(2000);
    document.getElementById('startBtn').disabled = true;
    document.getElementById('stopBtn').disabled = false;
  };
</script>

4. Production Optimization Tips

1. Client-Side VAD (Voice Activity Detection): Filter out silent audio chunks in the browser before sending requests to minimize unnecessary AI execution costs.
2. Multilingual & Translation Parameters: The @cf/openai/whisper (base) model used in this article does not support the language or task parameters. If you need multilingual recognition or automated translation, switch to the @cf/openai/whisper-large-v3-turbo model, then specify language: "ko" or pass task: "translate" to translate audio directly into English text.
3. WebSocket Integration: Combine Workers AI with Cloudflare Workers WebSockets for bi-directional real-time transcription feedback.

Summary and Conclusion

Combining Cloudflare Workers AI with Whisper transforms how developers build and maintain speech recognition pipelines.

[!NOTE]

1. Execute Whisper models with a single line of code env.AI.run('@cf/openai/whisper', ...) without managing GPU infrastructure.
2. Slice audio streams into 1–2 second chunks via the MediaRecorder API for instant real-time transcription feedback.
3. Eliminate server management costs and leverage global edge execution for minimal latency.

Deploying serverless Speech-to-Text at scale has never been easier or more cost-effective.