May 222009

Here are the slides I presented this week at the RCOS meeting. I got some good feedback about the meta data and search functionality. I broke rails on my development server this morning (accidentally chowned /var) so I’ve got to spend another 2-3 hours fixing that. I suspect I’ll have some code committed to git next week when I’ve got some basic upload stuff working.

May 202009

For the past week or so I’ve been thinking about the storage of video files, and how to implement a storage system that is fast, effecient, secure, and reliable. My big question here is to use a database or not to use a database. I’m likely using MySQL for everything else, but video files are a bit larger than you VARCHAR(255) or even larger than a photo gallery image I’d store in a database. I’m concerned that throwing video files, ranging from 100MB to 5GB in a database might start to slow things down… which I really don’t want to happen. On the other hand, I can’t afford to have video files laying around in violation of a database foreign constraint, which might happen if a user deletes a video and the file delete fails (maybe its in use).

I’ve done some work storing large datasets in MySQL, but no work storing >25MB in a single field. Of course there would have to be a seperate media table, to prevent a stray ‘SELECT * FROM videos’ fr0m crashing the whole thing. but that still doesn’t convince me it will be as fast as it needs to be.

Another nice plus on the filesystem side is the mod_flv and mod_h264 streaming plugins for Apache. I know you can write a version of the flv streaming module in PHP, but I don’t see it having the speed of an Apache plugin.

I think ideally I’d like to store videos in the database or using something besides the regular flat file approach. Using Ruby on Rails might make this a moot point from a programming perspective, attachment_fu easily handles both file and database datastores.

If you have any experience storing large files in a database, I’d love to hear about it.

May 142009

Here are some slides detailing the project objectives and timeline. I’m unsure if I’m going to be able to make it to the Friday meeting, but I’ll be continuing to make progress over the weekend. Hopefully mid/late next week there will be some code pushed to git for all to see!

May 132009

Hi everyone! I recently heard back from the RCOS leaders and my proposal has been accepted for development of an open source digital video server. In the next few days I’m going to be working out a rough schedule and getting my hands dirty with some code. As soon as I’ve got more to share I’ll post it here!