Basic idea of graph clustering is finding sets of "related" vertices in graphs. Graph clustering has many important applications in computing such as market research, pattern recognition, data analysis, image processing, etc., but due to growing sizes of graphs, even traditionally fast clustering methods can be computationally expensive for real-world graphs. As a first step towards understanding the graph clustering approach, we studied existing algorithms and explored new ideas in the world of graph clustering. We have implemented our algorithm parallelly. We have checked the efficiency and scalability of parallel algorithm on a modern MPI machine to work with large graphs and found that parallel algorithms give more runtime speedup than sequential algorithm.