Back to List

Beyond MediaRecorder: Building a High-Performance 4K Screen Recorder with WebCodecs & OPFS

5 min read

Beyond MediaRecorder: Building a High-Performance 4K Screen Recorder with WebCodecs & OPFS

Abstract: With the maturation of the WebCodecs API and Chrome’s transition to Manifest V3 (MV3), browser-based video processing is entering a new phase. This article explains how Screen Recorder Studio moved beyond the legacy MediaRecorder API to a fully streaming architecture built on Offscreen Documents, WebCodecs, and the Origin Private File System (OPFS), enabling stable 4K/60FPS recording without unbounded memory growth or catastrophic data loss.

🚀 Quick Links

1. Motivation: Why Move Beyond MediaRecorder?

For years, the MediaRecorder API has been the default choice for browser-based recording. Its appeal lies in its simplicity: provide a MediaStream, and receive a playable WebM file. However, when building a professional-grade screen recorder, we encountered architectural limitations that became impossible to ignore—especially at 4K resolution and long recording durations.

1.1 The Memory Trap (OOM)

In practice, MediaRecorder tends to accumulate encoded data in memory. Even when using the timeslice option to request periodic chunks, internal buffering behavior remains largely opaque and implementation-dependent.

  • Observed behavior: Recording 4K video for ~20 minutes frequently caused the tab process to exceed 3 GB of RAM.

  • Outcome: Chrome’s OOM killer terminates the process, resulting in an "Aw, Snap!" crash and complete loss of the recording.

While this may be acceptable for short or low-resolution captures, it is fundamentally incompatible with long-running, high-resolution workflows.

1.2 The Black Box Problem

MediaRecorder produces a pre-muxed container (typically WebM/Matroska), which severely limits control over the recording pipeline.

  • Late indexing: Seek tables (Cues) are commonly written only when recording finishes. A mid-session crash often leaves the file unseekable or entirely unusable.

  • Uncontrolled GOP structure: Developers cannot reliably enforce keyframe cadence or GOP size, complicating precise seeking and non-linear editing.

Together, these issues motivated a shift toward a fully streaming and transparent data pipeline, where each stage of capture, encoding, and storage is explicit and controllable.

2. Architecture Overview: MV3 Multi-Process Orchestration

Manifest V3 introduces strict constraints: Background Service Workers are ephemeral and cannot retain DOM access or MediaStream objects. To operate within these boundaries, we decomposed the system into three cooperating execution environments, connected through a message-based protocol.

System Layers

  1. Control Layer (Background Service Worker)

    • Role: Global state machine, lifecycle coordination, permission handling

    • Characteristics: Short-lived, no media or DOM access

  2. Ingestion Layer (Offscreen Document)

    • Role: Owns the MediaStream, extracts raw frames

    • Characteristics: Full DOM and media access, lifetime bound to the recording session

  3. Compute & Storage Layer (Dedicated Workers)

    • Role: Hardware-accelerated encoding (WebCodecs) and disk I/O (OPFS)

    • Characteristics: CPU- and I/O-intensive, isolated from UI and control logic

This separation ensures that heavy processing never blocks user interaction or extension control flow.

3. Implementation Details

3.1 Control: Orchestrating the State Machine

The Background Service Worker acts purely as a coordinator. It never touches video data directly; instead, it ensures that the required execution contexts are created and that recording transitions occur in a controlled sequence.

// src/extensions/background.ts (snippet)
import { ensureOffscreenDocument, sendToOffscreen } from '../lib/utils/offscreen-manager'

async function startRecordingFlow(options) {
  await ensureOffscreenDocument({
    url: 'offscreen.html',
    reasons: ['DISPLAY_MEDIA', 'WORKERS', 'BLOBS'],
  })

  await sendToOffscreen({
    target: 'offscreen-doc',
    type: 'OFFSCREEN_START_RECORDING',
    payload: { options },
  })
}

3.2 Ingestion: From MediaStream to VideoFrame

Within the Offscreen Document, we bypass MediaRecorder entirely. Instead, MediaStreamTrackProcessor is used to unwrap the video track into a stream of raw VideoFrame objects.

This marks a conceptual shift from file-oriented recording to real-time frame processing.

// src/extensions/offscreen-main.ts (snippet)

const processor = new (window as any).MediaStreamTrackProcessor({ track: videoTrack })
const reader = processor.readable.getReader()

const wcWorker = new Worker(
  new URL('../lib/workers/webcodecs-worker.ts', import.meta.url),
  { type: 'module' }
)

while (true) {
  const { value: frame, done } = await reader.read()
  if (done) break

  // Transfer ownership immediately to avoid frame accumulation
  wcWorker.postMessage({ type: 'encode', frame }, [frame])
}

Transferring ownership of each VideoFrame is critical: it ensures that frames are released as soon as they leave the ingestion layer, preventing memory pressure from accumulating over time.

3.3 Compute: Hardware-Accelerated Encoding with WebCodecs

The encoding Worker receives raw frames and feeds them directly into VideoEncoder. Unlike MediaRecorder, WebCodecs exposes detailed configuration knobs that are essential for predictable performance and downstream editing.

// src/lib/workers/webcodecs-worker.ts (snippet)
import { tryConfigureBestEncoder } from '../utils/webcodecs-config'

await tryConfigureBestEncoder(encoder, {
  codec: config?.codec ?? 'auto',
  width: config?.width ?? 1920,
  height: config?.height ?? 1080,
  framerate: config?.framerate ?? 30,
  bitrate: config?.bitrate,
  latencyMode: config?.latencyMode,
  hardwareAcceleration: config?.hardwareAcceleration,
})

encoder.encode(frame, { keyFrame: forceKey === true })

In practice, controlling keyframe cadence and encoder latency dramatically simplifies indexing, seeking, and later non-linear editing.

3.4 Storage: Streaming Writes with OPFS

Encoded chunks are never accumulated in memory. Instead, they are written incrementally to the Origin Private File System (OPFS) using FileSystemSyncAccessHandle, which provides a synchronous, high-throughput write API inside Workers.

Two append-only files are maintained:

  1. data.bin — the raw encoded bitstream
  2. index.jsonl — line-delimited metadata (timestamps, offsets, frame types)
// src/lib/workers/opfs-writer-worker.ts (snippet)

const root = await (self as any).navigator.storage.getDirectory()
const recDir = await root.getDirectoryHandle(`rec_${id}`, { create: true })

const dataHandle = await recDir.getFileHandle('data.bin', { create: true })
const sync = await (dataHandle as any).createSyncAccessHandle()

let offset = 0
const u8 = new Uint8Array(msg.buffer)
const start = offset

const written = sync.write(u8, { at: start })
offset += typeof written === 'number' ? written : u8.byteLength

await appendIndexLine(
  JSON.stringify({ offset: start, size: u8.byteLength, timestamp, type }) + '\n'
)

Because the storage model is append-only, partially recorded sessions remain recoverable even if the browser or operating system crashes mid-recording.

3.5 Assembly: Exporting to Standard Containers

EncodedVideoChunk objects are not directly playable. Recording and export are therefore treated as separate phases:

  • During recording: prioritize throughput and stability (capture → encode → write)

  • During export: read data.bin and index.jsonl, then multiplex the stream into a standard WebM or MP4 container

Deferring muxing avoids latency spikes and allows index structures to be generated with complete knowledge of the recording timeline.

4. Results and Observations

This architecture yields several tangible benefits:

  1. Flat memory usage — memory remains bounded by small processing buffers rather than total recording length
  2. Crash resilience — data is persisted incrementally, preserving nearly all captured content
  3. Editor-friendly output — predictable keyframes and explicit indexing enable fast seeking and precise editing

For modern, high-resolution browser recording, the combination of Offscreen Documents + WebCodecs + OPFS is not merely an optimization—it is a prerequisite for stability at scale.

Learn More

Tags

Screen Recorder Studio