Now Available on npm
Node.js 18+ Bun 1.0+

WebCodecs
for Node.js

Frame-level video encoding and decoding. No FFmpeg CLI. No subprocess overhead. Just TypeScript.

$ npm install node-webcodecs copy
$ bun add node-webcodecs copy
Documentation →

Server-side video in Node has been stuck in the FFmpeg era

FFmpeg filter_complex

// Dynamic text overlay with position animation
ffmpeg -i bg.mp4 -i overlay.png \
  -filter_complex \
  "[0:v][1:v]overlay=x='if(lt(t,5),10,100)'
  :y=10[v1];[v1]drawtext=text='Hello':
  fontsize=24:x=100:y=100:
  enable='between(t,2,8)'[v2]" \
  -map "[v2]" output.mp4

// Good luck debugging this

node-webcodecs

for await (const frame of decodeVideo(template)) {
  const t = frame.timestamp / 1_000_000;

  if (t >= 2 && t <= 8) {
    drawText(frame, customerName);
  }

  const x = t < 5 ? 10 : 100;
  compositeImage(frame, logo, { x, y: 10 });

  encoder.encode(frame);
}

Built for specific problems

Not "video processing in JS would be nice." These are the scenarios where WebCodecs architecture provides genuine engineering value.

01

Isomorphic Video Editors

Kapwing, Veed.io, Canva Video, Descript

Browser preview uses WebCodecs. Server export uses FFmpeg. Two codepaths. Two sets of bugs. Endless parity issues between what users see and what they get.

Same effects code runs browser and server. Visual parity guaranteed. One codebase to maintain.
// effects/textOverlay.js
// Runs identically in browser AND server

export function applyTextOverlay(frame, config) {
  const ctx = getContext(frame);
  ctx.font = config.font;
  ctx.fillStyle = config.color;
  ctx.fillText(config.text, config.x, config.y);
  return frame;
}
02

WebRTC Real-Time Composition

Riverside.fm, Streamyard, Squadcast

Recording multi-party calls with custom layouts requires screen capture (lossy) or separate native pipelines. Neither integrates with your Node WebRTC server.

Decode, composite, and re-encode in the same Node process handling your signaling. Real-time picture-in-picture without leaving JavaScript.
// In your Mediasoup server

participants.forEach(p => {
  const decoder = new VideoDecoder({
    output: frame => frames.set(p.id, frame)
  });
  p.videoTrack.pipe(decoder);
});

// Composite all participants in real-time
const composed = composePiP(frames, layout);
encoder.encode(composed);
03

Live Highlight Clipping

Twitch clippers, sports highlights, surveillance

Current pipelines analyze streams separately from recording. Timestamp handoff is lossy. Clips cut at keyframes, not the actual moment.

Single pipeline. Frame analyzed = frame encoded. No timestamp drift. Frame-accurate clips from live streams.
const decoder = new VideoDecoder({
  output: async (frame) => {
    const dominated = await detectKill(frame);

    if (dominated && !clipping) {
      startClip(frame.timestamp);
    }

    if (clipping) {
      clipEncoder.encode(frame);
    }
  }
});
04

Video Ingest Validation

Video platforms, asset management, UGC sites

ffprobe gives you container metadata. It doesn't tell you if frames are actually decodable, if there are frozen segments, or visual corruption.

Programmatic frame-by-frame validation. Detect black frames, frozen video, decode errors. Actual quality assurance, not metadata parsing.
async function validateVideo(file) {
  let frozen = 0, black = 0, lastHash;

  for await (const frame of decode(file)) {
    const hash = perceptualHash(frame);
    if (hash === lastHash) frozen++;
    if (isBlack(frame)) black++;
    lastHash = hash;
  }

  return { decodable: true, frozen, black };
}
05

Personalized Video Generation

Marketing automation, dynamic ads, video APIs

Template-based video with dynamic content means string-templating FFmpeg commands. Conditional logic becomes unreadable filter graphs.

Actual programming for video assembly. Variables, loops, conditionals. Per-frame encoding control. Debuggable, testable, version-controllable.
for await (const frame of decode(template)) {
  const t = frame.timestamp / 1e6;

  // Actual programming, not filter DSL
  if (customer.isPremium) {
    addGoldBorder(frame);
  }

  drawText(frame, customer.name, positions[t]);
  encoder.encode(frame, {
    keyFrame: t % 2 === 0
  });
}

What you actually get

Frame-Accurate Control

Encode, decode, seek to exact frames. Per-frame encoding parameters. Keyframe insertion where you want it.

Streaming Backpressure

Native integration with Node streams. encodeQueueSize for flow control. Memory-bounded processing.

Hardware Acceleration

Access to platform encoders/decoders. Queryable capability support. GPU path when available.

Browser API Parity

Same VideoEncoder, VideoDecoder, VideoFrame interfaces. Code runs both environments unchanged.

Explicit Memory

VideoFrame.close() for deterministic cleanup. No GC surprises. Predictable resource management.

TypeScript Native

Full type definitions. Autocomplete for codec configs. Catch errors at compile time.

Runs on Bun

Full compatibility with Bun runtime via N-API. Same package, both runtimes. No separate install.

HDR & Wide Gamut

BT.2020 primaries, PQ (HDR10) and HLG transfer functions. Encode HDR content without color space conversion.

ImageDecoder

Decode JPEG, PNG, GIF, WebP, BMP. Extract frames from animated GIFs with timing info.

Video as a first-class type
in Node.js

Frame-level access without leaving JavaScript.

$ npm install node-webcodecs copy
$ bun add node-webcodecs copy
View on GitHub