I’ve already committed a detailed blog post about that some time ago, so I’m not going to repeat myself - if you’re interested in Apache Hadoop (http://en.wikipedia.org/wiki/Apache_Hadoop) implementation “by Microsoft”, you should check their latest announcements (but don’t get too excited, it’s still in CTP phase) - http://blogs.msdn.com/b/windowsazure/archive/2012/10/24/getting-started-with-windows-azure-hdinsight-service.aspx
Yes, you’ve got it right - HDInsight is dedicated for Windows Azure, BUT as you can find here (http://www.microsoft.com/sqlserver/en/us/solutions-technologies/business-intelligence/big-data.aspx), there’s HDInsight Server version for Windows Server. Nice PDF with some technology blueprints can be found here: http://www.microsoft.com/en-us/download/details.aspx?id=35397 (and you can get the CTP via Web Platform Installer).
They’ve also started Hadoop SDK for .NET branch on Codeplex (http://hadoopsdk.codeplex.com/), but there’s nothing interesting there ATM ;/
If you feel like playing with some modern-approached distributed computing in .NET, but you don’t like fiddling with cloud stuff, there’s another project you can check out on your own. Naiad, the successor of famous Dryad. Version 0.1 was published last month and seems to be ready to rumble. Here are some links:
Naiad’s home page: http://research.microsoft.com/en-us/projects/naiad/
Nice tech report on Naiad: http://research.microsoft.com/pubs/174076/naiad.pdf
Naiad’s team blog: http://bigdataatsvc.wordpress.com/
To be honest, I’ve found it quite inspiring - but well, that’s me. I’m hooked on scalability and distributed computing (currently looking at Scala and Akka).
