Road conditions monitoring using semantic segmentation of smartphone motion sensor data

Emad Mahmood, Nizar Zaghden, Mahmoud Mejdoub


Many studies and publications have been written about the use of moving object analysis to locate a specific item or replace a lost object in video sequences. Using semantic analysis, it could be challenging to pinpoint each meaning and follow the movement of moving objects. Some machine learning algorithms have turned to the right interpretation of photos or video recordings to communicate coherently. The technique converts visual patterns and features into visual language using dense and sparse optical flow algorithms. To semantically partition smartphone motion sensor data for any video categorization, using integrated bidirectional Long Short-Term Memory layers, this paper proposes a redesigned U-Net architecture. Experiments show that the proposed technique outperforms several existing semantic segmentation algorithms using z-axis accelerometer and z-axis gyroscope properties. The video sequence's numerous moving elements are synchronised with one another to follow the scenario. Also, the objective of this work is to assess the proposed model on roadways and other moving objects using five datasets (self-made dataset and the pothole600 dataset). After looking at the map or tracking an object, the results should be given together with the diagnosis of the moving object and its synchronization with video clips. The suggested model's goals were developed using a machine learning method that combines the validity of the results with the precision of finding the necessary moving parts. Python 3.7 platforms were used to complete the project since they are user-friendly and highly efficient platforms.

Full Text:



M. Hamouda and M. S. Bouhlel, "Modified Convolutional Neural Networks Architecture for Hyperspectral Image Classification (Extra-Convolutional Neural Networks)," IET Image Processing, vol. 15, no. 2, pp. 305-313, 2021.

H. Li, "Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences," in IEEE International Conference on Multimedia and Expo (ICME), pp. 351-364, 2010.

M. Jasim, N. Zaghden and M. S. Bouhlel, "Identification of Collision Alert in Vehicle Ad hoc based on Machine learning," in IEEE International Conference on Computing (ICOCO), 2021.

S.V.-U. Ha ORCID, N.M. Chung, H.N. Phan ORCID and C.T. Nguyen, "TensorMoG: A Tensor-Driven Gaussian Mixture Model with Dynamic Scene Adaptation for Background Modelling," Sensors, vol. 20, no. 22, pp. 1-29, 2020.

S. Ammar, T. Bouwmans, M. Neji, and N. Zaghden, "Moving Objects Segmentation Based on DeepSphere in Video Surveillance," Academia, pp. 307-319, 2020.

B.D. Setiawan, U. Serdult, and V. Kryssanov, "A Machine Learning Framework for Balancing Training Sets of Sensor Sequential Data," Sensors, vol. 5, pp. 20-32, 2021.

N. Khalid, Y.Y. Ghadi, M. Gochoo, A. Jalal, and K. Kim, "Semantic Recognition of Human-Object Interactions via Gaussian-Based Elliptical Modeling and Pixel-Level Labeling," IEEE, pp. 111249-111266, 2021.

J. Sun, Y. Mao, Y. Dai, Y. Zhong and J.Wang,"Motion uncertainty-aware semi-supervised video object segmentation," Pattern Recognition vol.138,no109399 ,2023.

S.Ammar,T.Bouwmans,N.Zaghden,andM.Neji,"Deep detector classifier (DeepDC) for moving objects segmentation and classification in video surveillance," surveillance ,pp1490-1501 ,2020 .

Luca Greco, Pierluigi Ritrovato, Mario Vento, "On the use of semantic technologies for video analysis," IOS Press and the authors, vol. 2, no. 132, pp. 1-21, 2017.

B. D. Setiawana, M. Kovacs, U. Serdult, V. Kryssanov, "Semantic Segmentation on Smartphone Motion Sensor Data for Road Surface Monitoring," ScienceDirect, pp. 346-353, 2022.

V. A. K. Pawar, "Deep learning based detection and localization of road accidents from traffic surveillance videos," ScienceDirect, vol. 8, pp. 379-387, 2022.

O. Ronneberger, P. Fischer & T. Brox, "U-Net: Convolutional Networks for Biomedical Image Segmentation," Springers , vol. 3, no. 9351, pp. 234-241, 2015.

Y. Zhang et al., "Human Activity Recognition Based on Motion Sensor Using U-Net," IEEE, vol. 7, pp. 75213-75226, 2019.

L. Sigal et al., "Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion," International Journal of Computer Vision, pp. 4-27, 2010.

X. Li and D. W. Goldberg, "Toward a mobile crowdsensing system for road surface assessment," Computers, Environment and Urban Systems, vol. 69, pp. 51-62, 2018.

A. M. Abirami and V. Gayathrii, "A survey on sentiment analysis methods and," in IEEE, pp. 72-76, 2016.

S. Ioffe and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift," in International Machine Learning Society (IMLS), pp. 448-456, 2015.

O. Ronneberger, P. Fischer, and T. Brox, "U-Net: Convolutional networks for biomedical image segmentation," in Medical Image Computing and Computer-Assisted Intervention-MICCAI, vol. 9351, pp. 234-241, 2015.

L. Greco, "On the use of semantic technologies for video," Semantic Web Journal [Online]. Available:, Jan. 2021.

J. Ren, F. Xia, Y. Liu, and I. Lee, "Deep Video Anomaly Detection: Opportunities and Challenges," IEEE, pp. 1-5, 2021.

N. Zaghden, B. Khelifi, A.M. Alimi, and R. Mullot., "Text Recognition in both ancient and cartographic documents," arXiv preprint arXiv:1308.6309, pp. 98-101, 2013.

T. Zhao and Y. Wei, "A road surface image dataset with detailed annotations for driving assistance applications," Elsevier, vol. 12, pp. 23-50, 2022.

M. Otani, "Video Summarization using Deep Semantic Features," in Asian Conference on Computer Vision (ACCV), Oulu, Finland, 2016.

S. Ammar, T. Bouwmans, N. Zaghden, and M. Neji., "Moving objects segmentation based on deepsphere in video surveillance," in International Symposium on Visual Computing (ISVC), Cham Switzerland , 2019.

S.Saad., "Semantic Analysis of Human Movements in Videos," ACM Transactions on Multimedia Computing Communications and Applications (TOMM), vol .8 , no .3 , pp .141-148 ,2012 .

P.Gonçalves and M.Araújo., "Comparing and Combining Sentiment Analysis Methods," ACM Transactions on Internet Technology (TOIT), vol .14 , no .4 , pp .1-11 ,2014 .

R.Morais,V.Le,T.Tran,B.Saha,M.Mansour,and S.Venkatesh., "Learning regularity in skeleton trajectories for anomaly detection in videos," in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019.



  • There are currently no refbacks.

Copyright (c) 2023 Emad Mahmood, Nizar Zaghden, Mahmoud Mejdoub

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

ISSN: 2303-4521

Digital Object Identifier DOI: 10.21533/pen

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License