Date May 29, 2018
Speaker Julian Shun
Massachusetts Institute of Technology
Title Theoretically Efficient Parallel Graph Algorithms Can Be Fast and Scalable
Abstract

There has been significant interest in parallel graph processing recently due to the need to quickly analyze the large graphs available today. Many graph codes have been designed for distributed memory or external memory. However, today even the largest publicly-available real-world graph (the Hyperlink Web graph with over 3.5 billion vertices and 128 billion edges) can fit in the memory of a single commodity multicore server. Nevertheless, most experimental work in the literature report results on much smaller graphs, and the ones that use the Hyperlink graph are done in distributed or external memory. Therefore it is natural to ask whether we can efficiently solve a broad class of graph problems on this graph in memory.

With a graph of this size it is important to use theoretically-efficient parallel algorithms as even minor inefficiencies in the work or parallelism of an algorithm can lead to a significant increase in running time. This talk shows that theoretically-efficient parallel graph algorithms can scale to the largest publicly-available graphs using a single machine with a terabyte of RAM, processing them in minutes. We give implementations of theoretically-efficient parallel algorithms for 13 important graph problems. We also present the optimizations and techniques that we used in our implementations, which were crucial in enabling us to process these large graphs quickly. We show that the running times of our implementations outperform existing state-of-the-art implementations on the largest real-world graphs. For many of the problems that we consider, this is the first time they have been solved on graphs at this scale.

Bio Julian Shun is an assistant professor in Electrical Engineering and Computer Science at MIT. He is interested in the theory and practice of parallel computing, especially parallel graph processing frameworks, algorithms, data structures, and tools for deterministic parallel programming. He has received the ACM Doctoral Dissertation Award, CMU School of Computer Science Doctoral Dissertation Award, Miller Research Fellowship, Facebook Graduate Fellowship, and a best student paper award at the IEEE Data Compression Conference.
Resources