A comparative analysis of topic modelling methods on Twitter data related to Long COVID

This study compares five topic modeling methods - Latent Dirichlet Allocation (LDA), Latent Semantic Analysis (LSA), Non-negative Matrix Factorization (NMF), Top2Vec, and BERTopic - applied to 500,010 English-language tweets related to Long COVID collected throughout 2022. The goal is to evaluate each method's strengths and limitations using automatic metrics (coherence and diversity), human assessments (interpretability and relevance), and Large Language Model (LLM) evaluation. BERTopic achieved the highest coherence (0.6924) and diversity (0.796), effectively identifying nuanced and granular topics. However, its high number of topics (357) may reduce clarity for human readers. LDA and NMF offered more compact groupings with competitive coherence scores (0.5814 and 0.5875). Human raters preferred LDA, which scored highest in interpretability (4.11) and relevance (3.92). In contrast, BERTopic ranked highest in LLM-based evaluations, scoring 4.77 for interpretability and 4.13 for relevance, revealing a gap between human and machine perceptions. The findings suggest that BERTopic is best suited for exploratory analyses, while LDA remains effective for communicative, public-facing applications. No single model excels across all criteria, emphasizing the importance of aligning method choice with analysis goals. Future work should include multilingual data and dynamic topic modeling to capture evolving health discourse across platforms.