But I did the same thing 2.5 years ago with the BLOB Streaming Engine. In this blog I explain how you can retrieve data out of the database using the BLOB Streaming Engine and a simple URL of the form: http://mysql-host-name:8080/database/table/blob-column/condition
Where condition has the form: column1=value1&column2=value2&...
Now I have to ask myself the question: why did we not manage to generate more enthusiasm for the idea?
Many agree that we can learn more from failure than success, so here is my list of top reasons for this particular failure:
- Every idea has its time. In the last 2 years the awareness of NoSQL solutions has grown a lot, making RESTful and non-transactional storage and retrieval much better known and generally acceptable.
- We had no platform on which to launch the idea. Without a server distribution a plug-in does not have a chance of real exposure (this was not obvious back when we started making plug-ins). Percona Server and MariaDB now present such a platform. This is great for the whole community, so support them! :)
- Our software had not been proven in production. And this is one reason why building software based on an idea, instead of an actual project requirement is quite likely to fail.
- We did it with PBXT and not a the main stream storage engine which everyone is already using. The really exciting thing about HandlerSocket is that you can use it to grab data in your existing database. This will allow it to spread like wild fire in a dry forest.
- It is obvious to me that we at PrimeBase have a marketing problem! We have no clue how to get a message across to the public. It is really quite sad, and great technology like PBXT engine-level replication and BLOB streaming may die because of this. The following points also show lack of marketing skills - so next time you see him, hug your marketing guy! ;)
- By using the name BLOB Streaming Engine, we did not make it clear that this works for all kinds data, not just BLOBs. (OK, and MyBS no was a terrible name - PBMS not much better - but "HandlerSocket" will prove that it has nothing to do with the name!)
- We did not show benchmarks. For me it was obvious that retrieval would be significantly faster if it did not have to go through the SQL interface. Besides, as a developer I know you can easily manipulate benchmark results, so I reluctant to present them (although I do) for my own software.
But to developers it is not always obvious that giving it away for free does not automatically mean it will get used. So to my hacking compatriots: I hope this list will help you to do things better!
P.S. Congratulations Oracle on release of 5.5!
10 comments:
HandlerSocket mentioned that they handled
750k QPS, you probably missed out on this part of the marketing, also the name doesn't seem to be about fast queries. BLOBs doesn't make me think of fast queries. Not that 750k QPS is that much :) I ran a benchmark with 1.5M QPS in 2002.
You may be right in general, but IMHO storing BLOBs in databases is (or at least should be) a very niche/small market. This may have some relation why BLOB Streaming did not take off.
P.S. Storing BLOBs in databases has few use cases because in general storing them directly in filesystem is much easier to understand and maintain (and much cheaper), keeping only file paths in ACID database.
I think it's a combination of several things:
1) HandlerSocket got TONS of attention because of the published result of 750,000 qps. That's a very impressive number and it's guaranteed to attract a lot of attention
2) I see the name "BLOB Streaming" every time I come to this blog but I never bother to check it out, because I'm not using MySQL for streaming anything. Most users of MySQL don't use it for streaming. So if you name a product "BLOB Streaming", most people will just ignore it.
Was the performance as ridiculous as that reported for HandlertonSocket?
I would say that mentioning "blob's" and number 4 on your list were the most likely culprits.
I immediately associate blobs with images and binary data, not JSON documents. So you need to explain that BLOBs can be text too, but at that time you already lost part of your audience.
But having to use a new storage engine is the really killer for me. Everybody knows InnoDB, most of us run it. So adding HandlerSocket is a no brainer.
I agree that part of the reason for the lack of community uptake might be in the branding and focus on BLOBs. Heck, even I assumed that this was all it was about :)
And sure, being included as part of a larger distribution certainly helps to gain visibility - I assume you are currently experiencing the same with the inclusion of PBXT in MariaDB.
But you're not alone - JD's excellent mod_ndb Apache module (http://code.google.com/p/mod-ndb/) seems to suffer from a similar problem. Great idea and technology, but adoption has been quite low, as far as I can tell.
1. Marketing
2. Compelling product to market
3. Production case with numbers to reinforce compelling product to market
There is a long list of great products that never see the recognized success they probably deserve. I can't comment if yours is in this list.
Single user development over years is difficult for ongoing exposure and uptake.
If I remember correctly, Yoshinori-san moved to a new company, built on existing good work, deployed it quickly into an existing high volume production environment with compelling numbers of improvement. That's success.
Keep going.... InnoDB did not make it in one day as well !
Was the performance as ridiculous as that reported for HandlertonSocket?
Post a Comment