Ascii Video Player

A video player implemented using shaders

Implementing ASCII-style image processing can be done in two ways: brightness mapping or structure based. Here, I’ve chosen the former. My goal is to process videos in real time within the browser. While the result is less refined compared to the structure-based, it is simpler and more efficient to implement. For the latter, there is a research paper: Structure-based ASCII Art

Implementation steps:

1. Preprocess character textures

export const CHARSETS = {
  default: " .:-=+*#@",
  simple: " .:*#",
  detailed:
    " .'`^\",:;Il!i><~+_-?][}{1)(|/tfjrxnuvczXYUJCLQ0OZmwqpdbkhao*#MW&8%B@$",
};

2. Luminance calculation

float brightness = 0.3 \ R + 0.59 \ G + 0.11 \ B

3. Map the luminance value to the coordinates of the character texture.

  float charIndex = floor(brightness * (asciiAtlasSize.x - 1.0) + 0.5);
    vec2 asciiUv = vec2(
      (charIndex + cellUv.x) / asciiAtlasSize.x,
      cellUv.y
    );

  vec4 asciiChar = texture2D(tAsciiAtlas, asciiUv);

<a.mesh
  sizeScale
  color
  lighting
  brightness
  font
  effect
  ...
>
 {...primitive, material}
<a.mesh/>