Strata Conference and Hadoop World in New York

This week I get to spend a couple of days at the Strata Conference in New York. Combined with Strata is Hadoop World and that is why I chose to attend this conference instead of just waiting for the Strata conference in Santa Clara coming in February. Right now I am sitting in the keynote session where all of the major sponsors get to showcase their products. I don't really consider myself to be cynical but right now I find my eyes glazing over as I have seen way too many marketing pitches. But I won't let that keep me from giving my own marketing pitch.

So what is Hadoop and why does it matter? Hadoop is not something regular computer users will see in their homes. I'm sure there are several guys running it in their garage or basement, but that is more of an anomaly than normal. Hadoop is an open source project managed by Apache that allows users to take a bunch of cheap commodity computing hardware and combine it into a large data platform. It does this by making multiple copies (the default is three) so that the loss of hardware doesn't mean you lose data. You can also use the CPU's from all the computers in the system to process or look at the data stored in the system.

Hadoop matters to you in that companies can use it to process larger amounts of data. In the video game world, it means that companies can figure out what types of games you like to play and only market those types of games to you. It means they can figure out where you are getting stuck in a game and provide hints to help you get through those tough sections. It means that video game companies no longer rely on their gut to determine what their customers want and can analyze the data to truly see what is wanted.

This can be scary to some and exciting to others. Do we really want the companies we buy products from to know too much about us? That sounds a lot like Big Brother. However if I am in the market for a new car, automobile advertisements are seen as helpful. When I am not in the market for a car, those same advertisements are seen as annoying. Anything that can reduce the number of annoying advertisements I have to endure is beneficial. Now let us hope that is the outcome of Hadoop instead of just getting even more junk mail.