Identifying Blank Thumbnails

When a video is imported or uploaded, the system automatically generates 3 thumbnails of the video (beginning, middle, end). A lot of videos begin with a sort of fade up from black, and someone pointed out during my presentation last Friday, it might be neat if there was a way to avoid adding empty (all black) thumbnails.

To accomplish this, I compare the generated thumbnail with a plain black image that I store using RMagick, a Ruby wrapper for ImageMagick. Here’s the snippet of code that makes it happen:
def black(args = nil)
require 'RMagick'
src_black = Magick::ImageList.new(RAILS_ROOT+'/public/images/black.jpg')
source = Magick::ImageList.new(args[:path])
black = src_black.scale(source.columns, source.rows)
if source.difference(black)[1] < 0.01
return true
else
return false
end
end

The difference command only works when the images are of equal size, which is why I have the intermediary resize step. The difference method returns an array of values, and I found the ‘normalized mean error’ to be a pretty good indicator. When comparing a black thumbnail with the black image that value was on the order of 1 E -5 and a regular thumbnail generated a number in the range of .50-.90… so I opted for the 0.01 threshold. That should be enough to test for all black images, but still allow fairly dark shots thumbnails to pass through the system.

Thumbnailing Adjustments

The current thumbnail generation code runs in the same process as the web request to generate the thumbnails. Its much easier to extract a thumbnail image in the same code that creates the thumbnail object, and it worked pretty fast for a lot of my smaller test files. Now that I’ve started to work with larger and longer files, I’ve found the thumbnail generation process (which is handled via this command ffmpeg -i file.avi -y -deinterlace -f image2 -ss 00:12:34 image.jpg) has been too slow to be acceptable. I imagine that as video codecs get more complicated to compress video and maintain quality, the method to extract a single frame in the middle of a stream gets harder.

To fix this I’m going to have to re-write the thumbnail generation to be handled like video conversion and run as a background job. It unfortunately won’t present users with their thumbnails as quickly as I’d like (no one like to see a page saying “In Progress…”) but it should fix the long load times and subsequent timeouts caused by thumbnail generation.