Hadoop reduce method reuses objectsMonday, December 28th, 2009Over the holiday break I was playing around with creating a Map/Reduce job that would scan through all of the content items and then create a link graph. It was a fairly straightforward job. I would scan each content item for all of the hrefs and for each one would ... |
||
Archive for December, 2009
Hadoop and HBase in productionMonday, December 28th, 2009At the beginning of December, I ported several portions of the ReadPath infrastructure from MySQL to HBase with Hadoop. So far it's been a complete success. At first I had focused most of my attention on HBase, seeing it as a way to scale systems beyond a single MySQL instance without ... |
||