Saturday, February 06, 2010 12:53 AM
In Part 1 of this series, I mentioned that I really don’t see any need for RDBMS’s and I’ve yet to find a developer that can prove me wrong, however, there is one thing that I’ve yet to solve – that is reporting, although I don’t necessarily think its a problem. There is always the comment that you can’t report on databases that are not relational, how true is that? Very true! oh wait – not really!
Read David’s article – about how it all started.
However, SQL Server did not start at the latest version with all the bells and whistles, it started really small with probably nothing and it was built upon as demands are made. Therefore, I’m leaning towards popularity, I believe that if the concept of OODBMS or KVS (object oriented database management systems or key value stores) get popular enough, tools will be built that will report on data as you need, it probably won’t be easy but as any other technology evolution – it will evolve!
Reporting is not a show stopper for me and shouldn’t be for anymore, what are reports and what are they used for anyway; It’s more likely just a different way of looking at your existing data – there is no stopping you from writing a program that can create similar reports from a NOSQL database. I know you are probably thinking that I must be nuts because there won’t be any relationships, or inbuilt calculations, joins, functions, aggregations and etc. but think about it for a second, does it really need to be that way? Come 'on, lets say you want to get total orders per day in your so call report for the big guy at the 114th floor, shouldn’t you just have an entry that looks similar to [date, totalorders] in your NOSQL solution and simply increment totalorders when you receive a new order or increment the outstanding balance once a withdrawal is made, the whole idea of calculating totalorders after the fact is archaic, calculating it in real time is as or if not more efficient, accurate and up to date.
Lets take another real time example, COMCAST, yes I said COMCAST, I rented my house out and got a town home in the city (Atlanta) so that I can live the city life, the move went well and everything was needy and greedy but however, few weeks later my business internet stopped working, after calling Comcast and jumping hoops and feeding their hamsters, it came to my attention that my account wasn’t synced, my payments have been going to my home address – ha, problem #1! Secondly, even after the correction, it got disconnected twice again after that, to cut this story short, it was all because the system that feeds their accounting system receives data based on a reporting model and my changes weren’t their, they claim it takes up to a week to apply a balance, a week? Holy smokes – it felt like 1970! Now, if they were using a NOSQL solution and my account was balanced in real time, all other systems and departments will see the data in real time – there will be no need to wait for a scheduled generated report.
I’ve also been working on a secret analytics tool with a bang but guess what, building the system to collect the data was easy, now-storing and reporting on the data is the most painful process. As a matter of fact 70% of my time so far has been spent on storing, aggregating and reporting on the data – hence the reason why Google analytics has a 24hr lag, well and other reasons. In short, using an RDBMS has only made my application complex and over a year to build, with a NoSQL solution – it would have been so much easier.
NOSQL(http://en.wikipedia.org/wiki/NoSQL) – is the key to getting up and going in a matter of minutes.
Don’t get me wrong, I’m not really saying jump up and ditch your relational databases, I’m simply saying pick the right tool for the job. I build several web applications for my clients throughout the year and I can pretty much count with one hand of how many of them ask for some sort of report that I couldn’t imagine how to generate with an NoSQL solution. Therefore, NoSQL may not be the way to go for the next SalesForce but it sure could be the way to go for your new FaceBook competitor. The right tool for the job – heck, you can even start with a NoSQL solution and migrate to a relational database if you need to later, key point here is you will get up and running quickly and don’t have to worry about schemas and relationships.
You probably going - Ok shut up, get the point, so what are you going to do about it? Well, this is what I want to do, I want to build a NoSQL data store and provide it to the .NET community – it might not be all super powerful but it will be functional and that is as much as I can do to get the word out. I’ll work on two types of NoSQL solution for .NET a KVS and OODBMS – starting with the KVS first. Richard has a list here - http://www.metabrew.com/article/anti-rdbms-a-list-of-distributed-key-value-stores of key-value stores but as you can see, your .NET options are limited. I’m going to start with a KVS because I don’t want to get distracted with the underlying granularity of serializing custom objects and indexing – although I do have a pretty good idea of how I’ll write it but let’s start small – a prototype!
So how do I plan on going about building this KVS store and use it; it’s pretty simple - quickly, I’ll build a light weight prototype version of it and start using it in the real world. I’ll probably start with a simple website and this blog and my other sites and the rest of the world. Oh and it will probably need some sort of distributed capability. But in essence, this is my vision in pseudo code:
using(Database db = new Database("dotnetclr.db")){
db.Put("FirstName", "Rydal");
db.Close();
}
See how simple that is! Expect a whole lot more post on my progress in building my version of a key-value store in C#. RDBMS have done their time, we should move on!