MapReduce and K-Means Clustering Method for Long Text Summarization on Large Language Model

The rapid growth of digital information has resulted in an explosion in the volume of online text documents, ranging from news articles, and financial reports, to scientific papers. This condition triggers the need for an automated summarization system that can present important information in a concise and easy-to-understand manner. The application of Large Language Model (LLM) based summarization models on long documents still faces limited input length challenges. This paper proposes a long text summarization method based on LLM by combining MapReduce and K-means Clustering. Qwen2.5-7B model Instruct is used with Low-Rank Adaptation-based instruction fine-tuning technique. Long documents are divided into chunks, then converted into embedding and clustered using the K-Means algorithm. The clustering results are used to select the most relevant information representation, which is then summarized at the Map stage. The partial summaries are then merged and re-summarized at the Reduce stage to produce one final summary. This approach is capable of processing documents that exceed the token limit of LLM. To evaluate the summarization results, the ROUGE (Recall-Oriented Understudy for Gisting Evaluation) metric is used. An experimental evaluation was conducted on banking sector documents in the BRI dataset and showed that the MapReduce and Clustering method significantly improved performance over the direct truncation approach. Our approach achieves ROUGE-1, ROUGE-2, and ROUGE-L scores of 0.416,0.118, and 0.219 respectively compared to direct truncation scores of 0.320,0.090, and 0.168.