Friday, July 27, 2007

BLOB streaming engine (MyBS), version 0.5 Alpha released!

With some effort just before my holiday, I have managed to complete the release of the next version of MyBS, the BLOB streaming engine for MySQL.

This version includes all the basic functionality required to stream BLOB data in and out of MySQL tables.

The main features are:
  • Uploading of BLOB data directly into the database using HTTP PUT or GET methods.

  • Downloaded of BLOB data directly from the database using HTTP GET.

  • BLOB size may exceed 4GB - theoretical BLOB size limit of 256 Terabytes.

  • BLOBs are stored in a repository which manages references from other storage engine tables.

  • BLOBs are referenced by a URL.

  • URLs referencing BLOBs in the repository have a unique access code, for security.

  • The theoretical maximum repository size is 4 Zettabytes (2^72 bytes) per database.

  • The server-side streaming API allows any storage engine to store BLOB data in the repository.

  • MyBS system tables provide a view of the BLOBs and associated references in the repository.
MyBS works together with the PBXT transactional storage engine, version 0.9.88, which supports the MyBS streaming API. Both engines can be downloaded from: http://www.blobstreaming.org/download.

Documentation for MyBS is also available. It includes details about all features so far, and some examples of use: http://www.blobstreaming.org/documentation.

If you try out the new engine, I'd like to hear from you. Any comments, questions and bug reports can be sent directly to me.

4 comments:

Xaprb said...

Congratulations on the release.

Ben said...

This is really exciting news! I can't wait to give this a shot.

Nils said...

Is it possible to stream multiple blobs out of the database in parallel via HTTP GET?

Paul McCullagh said...

Hi Nils,

Yes. The streaming engine can handle many connections GET'ing and PUT'ing BLOBs in parrallel.

My performance tests show that the throughput increases with the number of connections, but I have not tested more than 16 connections yet.