Nowadays, many sources of information that can be used by the government to improve the problem handling in society. One alternative that can be used is gathering feedback from society through social media. Twitter has become a popular social media based on microblogging that allows the user to express their opinion and condition that happen and occur around them. Along with the massive number of users that express their feedback on Twitter, there is so much tweet that we must analyze to gather the citizen's complaint data manually, which is not an easy task to do. In this research, we offer text mining using Naïve Bayes classifier for classifying tweet automatically so that we can classify public's feedback to the government much faster. Also, from the classification, we can give a prediction to the government about what society is thinking about government, based on the keyword that we use. This research use 3000 tweet dataset regarding feedback for Pemerintah Kota Surabaya (Surabaya's City Government) that contains complaint or non-complaint tweet. The dataset was split into 160 data of complaint tweet and 140 non-complaint tweets. This research generates systems that can classify tweet automatically with 82.5% of accuracy.