TY - GEN
T1 - Web scraping for automated water quality monitoring system
T2 - 2016 International Seminar on Intelligent Technology and Its Application, ISITIA 2016
AU - Budiarti, Rizqi Putri Nourma
AU - Widyatmoko, Nanang
AU - Hariadi, Mochamad
AU - Purnomo, Mauridhi Hery
N1 - Publisher Copyright:
© 2016 IEEE.
PY - 2017/1/20
Y1 - 2017/1/20
N2 - The need for a better online water quality monitoring system to control and provide drinking water treatment processes is proven. Water utility company in Surabaya that called as PDAM Surabaya has few reservoirs in their water supply system which are being monitored by WTW IQ SensorNet 2020 XT. However, on that sensor devices while it can provide some of the water quality parameters value information but the sensor is passive and the internal data is still stored in the sensor itself. To solve the problem, we proposed an application of data logger to manage data collections online water quality monitoring system by using web scraping. This application built by using Python language, the application is able to collect data from the sensor by using web scraping. We have utilized Beautiful Soap library and store the data in SQL. In these research, our experiment shows the performance of the application by measuring the accuracy of data that has been scraped from sensors are around 99%, the error rate less than 1%, and MSE (Mean Square Error) around 0.35%. We also calculated the growth of the data size that is gradually increasing and also measured the correlation between the data size with the delay which shows the value of our data is around 0.0000002739. Its means that our application is able to work in real-time and the delay is not affected by the size of data.
AB - The need for a better online water quality monitoring system to control and provide drinking water treatment processes is proven. Water utility company in Surabaya that called as PDAM Surabaya has few reservoirs in their water supply system which are being monitored by WTW IQ SensorNet 2020 XT. However, on that sensor devices while it can provide some of the water quality parameters value information but the sensor is passive and the internal data is still stored in the sensor itself. To solve the problem, we proposed an application of data logger to manage data collections online water quality monitoring system by using web scraping. This application built by using Python language, the application is able to collect data from the sensor by using web scraping. We have utilized Beautiful Soap library and store the data in SQL. In these research, our experiment shows the performance of the application by measuring the accuracy of data that has been scraped from sensors are around 99%, the error rate less than 1%, and MSE (Mean Square Error) around 0.35%. We also calculated the growth of the data size that is gradually increasing and also measured the correlation between the data size with the delay which shows the value of our data is around 0.0000002739. Its means that our application is able to work in real-time and the delay is not affected by the size of data.
KW - Web scraping
KW - online real-time
KW - water monitoring sensor
UR - http://www.scopus.com/inward/record.url?scp=85016762537&partnerID=8YFLogxK
U2 - 10.1109/ISITIA.2016.7828735
DO - 10.1109/ISITIA.2016.7828735
M3 - Conference contribution
AN - SCOPUS:85016762537
T3 - Proceeding - 2016 International Seminar on Intelligent Technology and Its Application, ISITIA 2016: Recent Trends in Intelligent Computational Technologies for Sustainable Energy
SP - 641
EP - 648
BT - Proceeding - 2016 International Seminar on Intelligent Technology and Its Application, ISITIA 2016
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 28 July 2016 through 30 July 2016
ER -
