http://philippeadjiman.com/blog Just Another Blog About Geek Stuff, by Philippe Adjiman Sat, 20 Feb 2010 17:47:26 +0000 http://backend.userland.com/rss092 en I'm regularly using the excellent google collections library (now final and part of the more general guava libraries). One of the data structure I'm using the most is probably the multiset (a.k.a bag). But most of the time, when I need a multiset to track the number of occurrences of particular entries, ... http://philippeadjiman.com/blog/2010/02/20/a-generic-method-for-sorting-google-collections-multiset-per-entry-count/ Lately I wrote a post on how to build a relevant real time search engine prototype in few hundreds lines of code.  Using a tailored ranking algorithm based on link popularity in twitter,  I showed that the prototype was able to return very relevant answers in response to very hot ... http://philippeadjiman.com/blog/2010/02/06/what-are-the-10-most-cited-websites-on-twitter-when-tweeting-about-hot-trends/ Welcome to the fourth issue of the Hadoop Tutorial Series. Combiners are another important Hadoop's feature that every hadoop developer should be aware of. The primary goal of combiners is to optimize/minimize the number of key value pairs that will be shuffled accross the network between mappers and reducers and ... http://philippeadjiman.com/blog/2010/01/14/hadoop-tutorial-series-issue-4-to-use-or-not-to-use-a-combiner/ Note: This post has been updated with a code working for hadoop 0.20.1. In this 3rd issue of the hadoop tutorial series, we'll speak about a very simple but very useful hadoop's feature: counters. Even if you have never defined any counters in hadoop, you can see some of them each time ... http://philippeadjiman.com/blog/2010/01/07/hadoop-tutorial-series-issue-3-counters-in-action/ By the end of the post you'll find the code along with a small command line JAVA program to play with, but let me first describe the specifications of the real time search engine prototype that I'm targeting here. Basically it should take as input a  search query and return as ... http://philippeadjiman.com/blog/2010/01/06/how-to-build-a-relevant-real-time-search-engine-prototype-in-few-hundred-lines-of-code/ In the Issue #1 of this series, we set up the "learning playground" (based on the Cloudera Virtual Machine) in order to enjoy hands-on learning experiences around Hadoop. In this issue, we'll use our playground to investigate the partitioning features offered by Hadoop. What is it all about? As you may know, ... http://philippeadjiman.com/blog/2009/12/20/hadoop-tutorial-series-issue-2-getting-started-with-customized-partitioning/ This is the first post of a series of small hadoop tutorials introducing progressively core hadoop functionnalities. You might be interested in that series if you recognized yourself in one or more of the following points : You've heard about the basics of MapReduce (else check the links that I ... http://philippeadjiman.com/blog/2009/12/07/hadoop-tutorial-part-1-setting-up-your-mapreduce-learning-playground/ I recently had to build quickly a prototype of recommendation engine for a promising start-up company. I wanted to first test state of the art collaborative filtering algorithms before to build a customized solution (potentially on top of those algorithms). Most importantly, I wanted to be able to compare ... http://philippeadjiman.com/blog/2009/11/11/flexible-collaborative-filtering-in-java-with-mahout-taste/ If you need to parse the tokens n-grams of a string, you may use the facilities offered by lucene analyzers. What you simply have to do is to build you own analyzer using a ShingleMatrixFilter with the parameters that suits you needs. For instance, here the few lines of code ... http://philippeadjiman.com/blog/2009/11/02/writing-a-token-n-grams-analyzer-in-few-lines-of-code-using-lucene/ There are many tools out there to build more or less quickly any kind of graphs. Depending on your needs a tool may be more suited than another. When it comes to draw graphs from a set of generated coordinates, I love the simplicity of gnuplot. Let's see together a simple ... http://philippeadjiman.com/blog/2009/10/26/drawing-the-long-tail-of-a-zipf-law-using-gnuplot-java-and-moby-dick/