Producing a lot of data doesn't bring value by itself, even if data actually makes sense. Storing a lot of data doesn't bring value by itself either. The ability to process all this data is the key to bring some actual value. What is more, the processing has to: be efficient be ready for continuous adjustments, enhancements, improvements & further development Honestly, if you're dealing with relatively simple data & straightforward analysis, you don't really need sophisticated tools: we've found out that in case of Hadoop (or to be more precise: Hadoop over HDFS), a simple abstraction layer like Hive…
All posts in Hadoop
Episode #1: http://gebski.tumblr.com/post/62260870845/bakin-bakin-bakin-logging-for-enterprise-scaleEpisode #2: http://gebski.tumblr.com/post/63404293910/bakin-bakin-bakin-logging-for-enterprise-scale Let’s get back to our enterprise logging scenario: we’ve kinda already sketched the overall topology: we’re gonna collect events using MOM (RabbitMQ), process them in Twitter Storm (aka Apache Storm) and finally we’re going to put the processed events in distributed document database (Cassandra) and file storage (HDFS). All of that to get rid of all good SQL Niiiiice, and surely we can enumerate many pros (I believe I did it in prior posts), but…