Design and Implementation of Artificial Intelligence-based Real-time Pothole Detection for IoT Smart Infrastructure

  • Rahardhita Widyatra Sudibyo Department of Electronic Engineering, Politeknik Elektronika Negeri Surabaya
  • Akhmad Alimudin Department of Multimedia Creative Technology, Politeknik Elektronika Negeri Surabaya
https://doi.org/10.25139/inform.v8i2.6221
Abstract views: 131 , PDF downloads: 138
Keywords: IoT, AI, Pothole Detection, CNN, Mobilenet SSDV2

Abstract

The Internet of Things (IoT) has been extensively deployed for Smart Cities due to its ability to process many different and heterogeneous end systems. The IoT innovation encourages artificial intelligence applications to process data. In Smart City infrastructure, the road is a critical component of transportation infrastructure that supports the economic, social, and cultural things of community life and various aspects of community life. Road conditions affect a variety of community activities. Good roads enhance comfort and support local businesses. However, many roads remain in bad condition, such as potholes. Various methods have been attempted to identify potholes, especially the two-dimensional imaging method. This paper proposes the real-time Artificial Intelligence detection of potholes using the Convolutional Neural Network (CNN), which leverages the Edge Tensor Processing Unit (TPU) with the MobileNet SSD v2. The system was set up on a Jetson Nano with a few extras, including a camera and GPS, to support the IoT infrastructure. Evaluation for the model consists of device implementation, model evaluation, GPS position deviation, and on-road implementation. The effectiveness is confirmed through experiments using a system test-bed that generates ideal mAP off 0.22 and recall values.

 

Author Biographies

Rahardhita Widyatra Sudibyo, Department of Electronic Engineering, Politeknik Elektronika Negeri Surabaya

 

 

Akhmad Alimudin, Department of Multimedia Creative Technology, Politeknik Elektronika Negeri Surabaya

 

 

References

B. B. V. Rao, H. G. Chandrakanth, "IoT based Machine Learning Automation Algorithm for Controlling the Industrial Loads", in Int. J. of Intelligent Engineering and Systems, Vol.14, No.4, pp. 146-156, Aug. 2021.

S. Deng, H. Zhao, W. Fang, J. Yin, S. Dustdar and A. Y. Zomaya, "Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligence", in IEEE Internet of Things Journal, vol. 7, no. 8, pp. 7457-7469, Aug. 2020.

I. R. Putra, “Ministry of Public Works: 3.338 Holes along the North Coast Road in Java Island”, https://www.merdeka.com/uang/kemen-terian-pu-ada-3338-lubang-di-sepanjang-jalur-pantura.html. Accessed 18 Nov. 2022.

Md. S. Arman, Md. M. Hasan, F. Sadia, A. K. Shakir, K. Sarker, and F. A. Himu, "Detection and Classification of Road Damage Using RCNN and Faster R-CNN: A Deep Learning Approach", in Cyber Security and Computer Science. ICONCS 2020. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 325. Springer, July 2020.

R. Mukherjee, H. Iqbal, S. Marzban, A. Badar, T. Brouns, S. Gowda, E. Arani, and B. Zonoos, "AI Driven Road Maintenance Inspection," Proc. 27th ITS World Congress, June 2021.

S. Naddaf-Sh, M. M. Naddaf-Sh, A. R. Kashani, and H. Zargarzadeh, "An Efficient and Scalable Deep Learning Approach for Road Damage Detection," Proc. 2020 IEEE Int. Conf. Big Data, pp. 5602–5608, 2020.

Z. Liu, W. Wu, X. Gu, S. Li, L. Wang, and T. Zhang, "Application of combining yolo models and 3d gpr images in road detection and maintenance," Int. J. Remote Sens., vol. 13, no. 6, pp. 1–18, 2021.

Z. Pei, R. Lin, X. Zhang, H. Shen, J. Tang, and Y. Yang, "CFM: A Consistency Filtering Mechanism for Road Damage Detection," Proc. 2020 IEEE Int. Conf. Big Data, pp. 5584-5591, 2020.

D. Arya, H. Maeda, S. K. Ghosh, D. Toshniwal, A. Mraz, T. Kashiyama, and Y. Sekimoto, "Deep learning-based road damage detection and classification for multiple countries," Int. J. Autom. Constr., vol. 132, 2021.

Y. Liu, X. Zhang, B. Zhang, and Z. Chen, "Deep Network for Road Damage Detection," Proc. 2020 IEEE Int. Conf. Big Data, pp. 5572–5576, 2020.

F. Kortmann, K. Talits, P. Fassmeyer, A. Warnecke, N. Meier, J. Heger, P. Drews, and B. Funk, "Detecting Various Road Damage Types in Global Countries Utilizing Faster R-CNN," Proc. 2020 IEEE Int. Conf. Big Data, pp. 5563–5571, 2020.

T. Hascoet, Y. Zhang, A. Persch, R. Takashima, T. Takiguchi, and Y. Ariki, "Faster RCNN Monitoring of Road Damages: Competition and Deployment," Proc. 2020 IEEE Int. Conf. Big Data, pp. 5545–5552, 2020.

D. Arya, H. Maeda, S. K. Ghosh, D. Toshniwal, and Y. Sekimoto, "RDD2020: An annotated image dataset for automatic road damage detection using deep learning," Int. J. Data in Brief, vol. 36, 2021.

A. Mahardika, Y. Sari, and C. Dewi, "Sistem Temu Kembali Citra Lubang Jalan Aspal Berdasarkan Tingkat Kerusakan Menggunakan Ekstraksi Fitur Gray Level Co-occurrence Matrix", in Jurnal Pengembangan Teknologi Informasi dan Ilmu Komputer, vol. 2, no. 10, pp. 3811-3821, Feb. 2018.

H. Maeda, Y. Sekimoto, and T. Seto, "Lightweight Road Manager: Smartphone-based Automatic Determination of Road Damage Status by Deep Neural Network", in MobiGIS '16: Proc. of the 5th ACM SIGSPATIAL International Workshop on Mobile Geographic Information Systems, pp. 37–45, 2016.

I. D. Pratama, H. Mahmudah, and R. W. Sudibyo, "Design and Implementation of Real-time Pothole Detection using Convolutional Neural Network for IoT Smart Environment", in Proc. 2021 Int. Electron. Symp. IES pp. 675–679, 2021.

Z. S. Hernanda, H. Mahmudah, and R. W. Sudibyo, "CNN-Based Hyperparameter Optimization Approach for Road Pothole and Crack Detection Systems", in Proc. 2022 World AI IoT Congress pp. 549–554, 2022.

K. E. An, S. W. Lee, S.-K. Ryu, and D. Seo, "Detecting a Pothole using Deep Convolutional Neural Network Models for an Adaptive Shock Oserving in a Vehicle Driving", in Proc. IEEE International Conference on Consumer Electronics (ICCE), pp. 1-2, 2018.

V. Pereira, S. Tamura, S. Hayamizu, and H. Fukai, "A deep learning based approach for road pothole detection in Timor Leste", in Proc. IEEE International Conference on Service Operations and Logistics, and Informatics (SOLI), pp. 279-284, 2018.

L. K. Suong and J. Kwon, "Detection of Potholes Using a Deep Convolutional Neural Network", in Int. J. UCS, vol. 24, no. 9, pp. 1244– 1257, 2018.

Y. Y. F. Panduman, A. R. A. Besari, S. Sukaridhoto, R. P. N. Budiarti, R. W. Sudibyo, and F. Nobuo, “Implementation of Integration VaaMSN and SEMAR for Wide Coverage Air Quality Monitoring”, in Int. Journal TELKOMNIKA, vol. 16, no. 6, p. 2630, Dec. 2018.

Z. Zhao, P. Zheng, S. Xu and X. Wu, "Object Detection with Deep Learning: A Review", in IEEE Transactions on Neural Networks and Learning Systems, vol. 30, no. 11, pp. 3212-3232, Nov. 2019.

W. S. E. Putra, A. Y. Wijaya, and R. Soelaiman, “Klasifikasi Citra Menggunakan Convolutional Neural Network (CNN) pada Caltech 101”, in Jurnal Teknik ITS, vol. 5, no. 1, pp. A65-A69, 2016.

Aparna, Y. Bhatia, R. Rai, V. Gupta, N. Aggarwal, and A. Akula, "Convolutional Neural Networks based Potholes Detection using Thermal Imaging", in Journal of King Saud University - Computer and Information Sciences, Feb. 2019.

N. C. Thompson, K. Greenewald, K. Lee, and G. F. Manso, "The Computational Limits of Deep Learning", in arXiv:2007.05558v1, July 2020.

NVIDIA, Jetson Nano Developer Kit, https://developer.nvidia.com/ embedded/jetson-nano-developer-kit. Accessed 18 Nov. 2022.

A. Reuther, P. Michaleas, M. Jones, V. Gadepally, S. Samsi, and J. Kepner, "Survey of Machine Learning Accelerators", in arXiv:2009.00993v1, Sep. 2020.

W. Li and M. Liewig, "A Survey of AI Accelerators for Edge Environment", in Trends and Innovations in Information Systems and Technologies. WorldCIST 2020. Advances in Intelligent Systems and Computing, vol 1160. Springer, Cham, Jun. 2020.

A. Basulto-Lantsova, J. A. Padilla-Medina, F. J. Perez-Pinal, and A. I. Barranco-Gutierrez, "Performance Comparative of OpenCV Template Matching Method on Jetson TX2 and Jetson Nano Developer Kits", in Proc. 10th Annual Computing and Communication Workshop and Conference (CCWC). pp. 0812-0816, Jan. 2020.

Google Coral, Build Beneficial and Privacy Preserving AI, https: //coral.ai/. Accessed 18 Nov. 2022.

L. A. Libutti, F. D. Igua, L. Pi˜nuel, and L. D. Giusti, "Benchmarking Performance and Power of USB Accelerators for Inference with MLPerf", in 2nd Workshop on Accelerated Machine Learning (AccML), Spain 2020.

Coral Benchmark, Edge TPU Performance Benchmarks, https://coral.ai/docs/edgetpu/-benchmarks/. Accessed 18 Nov. 2022.

J. Huang, V. Rathod, C. Sun, M. Zhu, A. Korattikara, A. Fathi, I. Fischer, Z. Wojna, Y. Song, S. Guadarrama, et al, "Speed/accuracy trade-offs for modern convolutional object detectors", in Proc. of the IEEE conference on computer vision and pattern recognition, pp. 7310–7311, 2017.

Y. Zhang, H. Pheng, and P. Hu, "CS341 Final Report: Towards Real-time Detection and Camera Triggering", in 2017.

M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, "MobileNetV2: Inverted Residuals and Linear Bottlenecks", in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018.

B. Jacob et al., "Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference", in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2018.

T. Vincenty, "Direct and Inverse Solutions of Geodesics on the Ellipsoid with Application of Nested Equation", in Survey Review Ministry of Overseas Development Surrey, vol. XXIII. no. 176, Apr. 1975.

Apache JMeter, https://jmeter.apache.org/. Accessed 18 Nov. 2022.

Published
2023-06-06
How to Cite
Sudibyo, R. W., & Alimudin, A. (2023). Design and Implementation of Artificial Intelligence-based Real-time Pothole Detection for IoT Smart Infrastructure. Inform : Jurnal Ilmiah Bidang Teknologi Informasi Dan Komunikasi, 8(2), 125-131. https://doi.org/10.25139/inform.v8i2.6221
Issue
Vol. 8 No. 2 (2023)
Section
Articles

Copyright (c) 2023 Rahardhita Widyatra Sudibyo, Akhmad Alimudin

Creative Commons License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Authors who publish with Inform: Jurnal Ilmiah Bidang Teknologi Informasi dan Komunikasi agree to the following terms:

  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (CC BY-SA 4.0) that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal. 

  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.

  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.