Fast graph mining with HBase

Mining large graphs using distributed platforms has attracted a lot of research interests. Especially, large graph mining on Hadoop has been researched extensively, due to its simplicity and massive scalability. However, the design principle of Hadoop to maximize scalability often limits the efficiency of the graph algorithms. For this reason, the performance of graph mining algorithms running on top of Hadoop has not been satisfactory.In this paper, we propose Unicorn, a graph mining library on top of HBase, an open source version of Bigtable. Unicorn exploits the random write characteristic of HBase to improve the performance of generalized iterative matrix–vector multiplication (GIM–V), a core graph mining routine. Extensive experiments show that Unicorn outperforms its predecessors by an order of magnitude for a graph with 68 billion edges.