The rapid development of information technology and its applications with the emergence of internet media makes disseminating information more accessible, fast and creating huge data in any time. Government is one of important stakeholders that produce a big data every day. Big data is one combination with data analytics that plays an important role in data processing and new insight retrieving. In this study, text data from customers complaint regarding the services given by the Government organization namely Financial Monitoring and Development Agency was analyzed. Users can report various kinds of complaints related to the problems they experienced. In this study, new insights regarding the applications provided by the Government agency will be discussed. From the 15 thousand complaint data records, six groups of the most dominant complaint regarding the applications use were then categorized: SIMA applications, SIBIJAK applications, GDN applications, SADEWA applications, Lotus Notes, and Infrastructure. Latent Dirichlet Allocation (LDA) topic modeling with part-of-speech tagger techniques was used to disseminate information on the topics. The results showed that the SIMA application gave 52% of all complaints reports based on the method used. With the implementation of the LDA topic modeling, four topics were generated: complaints about using the SIMA application, the service and installation of the Lotus Notes and SADEWA application, and complaints related to the existing network infrastructure of Government Agency. In conclusion, inference LDA Topic modeling successfully provided insights to government organization regarding which aspects within organization that are needed to be improved.