A Survey and Experimental Review on Data Distribution Strategies for Parallel Spatial Clustering Algorithms

The advent of Big Data has led to rapid growth in the usage of parallel clustering algorithms that work over distributed computing frameworks such as MPI, MapReduce & Spark. An important step for any parallel clustering algorithm is the distribution of data amongst the cluster nodes. This step governs the methodology and performance of the entire algorithm. Researchers typically use random, or a spatial/geometric distribution strategy like kd-tree-based partitioning and grid-based partitioning, as per the requirements of the algorithm. However, these strategies are generic and are not tailor-made for any specific parallel clustering algorithm. In this paper, we give a very comprehensive literature survey of MPI-based parallel clustering algorithms with special reference to the specific data distribution strategies they employ. We also propose three new data distribution strategies namely - Parameterized Dimensional Split for parallel density-based clustering algorithms like DBSCAN and OPTICS, Cell-based Dimensional Split for dGridSLINK, which is grid-based hierarchical clustering algorithm that exhibits efficiency for disjoint spatial distribution, and Projection-based Split, which is a generic distribution strategy. All of these preserve spatial locality, achieve disjoint partitioning, and ensure good data load balancing. The experimental analysis shows the benefits of using the proposed data distribution strategies for algorithms they are designed for, based on which we give appropriate recommendations for their usage.