Posted by: Morten Nobel-Jørgensen | November 12, 2011

Creating real time video ASCII art using KickJS and WebGL


Have you ever wondered how videos would look like they were rendered using only ASCII fonts? Well, here is the answer. I have created a post processing effects that transform a video feed into ASCII art.

Showtime

First let’s start with a little demo. You can either choose a live demo (reduced quality and length – or use your own video) or see the video of the whole thing:

 

Creating a good ASCII font

To create a nice ASCII effect, you obviously need an ASCII font (I used a 8×16). In my first attempt I simply sorted the font based on the number of black pixels in the font. One problem with this approach is that most ASCII characters are mostly white. To fix this problem I added the inverted ASCII characters as well. To reduce the size of the font to 256 characters, I discarded every second character. This gives me the following font (click on the ‘line’ below to see the full font).

The full source code for creating this font can be found on Gist:

https://gist.github.com/1343465

Creating the ASCII shader

KickJS Shader Editor

The shader is a fullscreen post processing effect (meaning that it has one input texture and expects the geometry to be a full screen quad). The shader does three things – all things done in the fragment shader:

  • First it pixelate the image into blocks of 8×16 (the size of an ASCII character). This is done by sampling the 8×16 neighbor pixels of the input texture based on the fragment position in screen space – and calculating the average color.
  • Lookup the correct ASCII char pixel based on the brightness of the average color. This is also done in screen space.
  • Multiply the ASCII char color with the average color. To improve the pixelated effect I also reduce the number of unique colors to 512 (8 for each channel).

The ASCII shader is created using KickJS Shader Editor and the source code for the shaders can be found here:

http://goo.gl/Jsne8

Creating video texture

Creating a video texture in WebGL is easy. This is done by simply updating a texture with a video element. To keep things simple, I do this every frame (in theory this should most likely be more every second frame – assuming that effect runs at 60 fps and video often is between 24-30 fps).

To keep things simple I use KickJS for creating the WebGL application. The full source code is only around 200 lines with only around 100 lines of WebGL related code.

The source code is available here:

https://github.com/mortennobel/KickJS/blob/master/example/video_ascii_art/ascii.js

Possible optimizations

Even though the shader works quite well a few things could be optimized:

  • Currently the pixelation is done in each pixel. This means that the exact same computation is done for every one of the 128 pixels in each ASCII character. However to solve this problem you would need to use a ‘render to texture’ approach (using two camera where one of them has a RenderTexture attached). In other words – it is certainly possible, but it will make the code slightly more complex.
  • Currently the shader darkens the image since it basically takes the average color and adds black pixels to it. A better approach would be to convert the RGB color into HSB/HSL or HSV color space and find the correct color taken the blackness into account.
  • A smarter algorithm for selecting the best 256 of 512 the characters when creating the ASCII font.
Advertisements

Responses

  1. […] Creating real time video ASCII art using KickJS and WebGL … Have you ever wondered how videos would look like they were rendered using only ASCII fonts? Well, here is the answer. I have created a post processing effects that transform a video feed into ASCII art. Source: blog.nobel-joergensen.com […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: