Object-based Classification of Natural Scenes Using Machine Learning Methods
Article Sidebar
Main Article Content
Abstract
The replication of human intellectual processes by machines, particularly computer systems, is known as artificial intelligence (AI). AI is an intelligent tool that is utilized across sectors to improve decision making, increase productivity, and eliminate repetitive tasks. Machine learning (ML) is a key component of AI since it includes understanding and developing ways that can learn or improve performance on tasks. For the last decade, ML has been applied in computer vision (CV) applications. In computer vision, systems and computers extract meaningful data from digital videos, photos, and other visual sources and use that information to conduct actions or make suggestions. In this work, we have solved the image segmentation problem for the natural images to segment out water, land, and sky. Instead of applying image segmentation directly to the images, images are pre-processed, and statistical and textural features are then passed through a neural network for the pixel-wise semantic segmentation of the images. We chose the 5X5 window over the pixel-by-pixel technique since it requires less resources and time for training and testing.
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
References
A. I. Khan and S. Al-Habsi, “Machine Learning in Computer Vision,” in Procedia Computer Science, 2020, vol. 167, pp. 1444–1451. doi: 10.1016/j.procs.2020.03.355.
P. A. Kragel, M. C. Reddan, K. S. LaBar, and T. D. Wager, “Emotion schemas are embedded in the human visual system,” Sci Adv, vol. 5, no. 7, 2019, doi: 10.1126/sciadv.aaw4358.
L. N. Ayton et al., “An update on retinal prostheses,” Clinical Neurophysiology, vol. 131, no. 6. pp. 1383–1398, 2020. doi: 10.1016/j.clinph.2019.11.029.
S. Taghizadegan, Essentials of Lean Six Sigma. 2006. doi: 10.1016/B978-0-12-370502-0.X5000-0.
N. V. Gavrilovskaya, V. P. Kuvaldin, I. S. Zlobina, D. E. Lomakin, and E. E. Suchkova, “Developing a robot with computer vision for automating business processes of the industrial complex,” in Journal of Physics: Conference Series, 2021, vol. 1889, no. 2. doi: 10.1088/1742-6596/1889/2/022024.
“What is Computer Vision? | IBM.”
J. Casadesús, D. V.-J. of integrative plant biology, and undefined 2014, “Conventional digital cameras as a tool for assessing leaf area index and biomass for cereal breeding,” Wiley Online Library.
G. Meyer, J. N.-C. and electronics in agriculture, and undefined 2008, “Verification of color vegetation indices for automated crop imaging applications,” Elsevier.
E. Hamuda, M. Glavin, E. J.-C. and electronics in agriculture, and undefined 2016, “A survey of image processing techniques for plant extraction and segmentation in the field,” Elsevier.
Z. Li et al., “Deep learning based cloud detection for medium and high resolution remote sensing images of different sensors,” Elsevier.
A. Prest, C. Leistner, J. Civera, … C. S.-2012 I. C., and undefined 2012, “Learning object class detectors from weakly annotated video,” ieeexplore.ieee.org.
H. Deng, D. C.-P. recognition, and undefined 2004, “Unsupervised image segmentation using a simple MRF model with a new implementation scheme,” Elsevier.
M. Helbich, Y. Yao, Y. Liu, J. Zhang, P. L.-E. international, and undefined 2019, “Using deep learning to examine street view green and blue spaces and their associations with geriatric depression in Beijing, China,” Elsevier.
Y. Sun, X. Wang, J. Zhu, L. Chen, … Y. J.-S. of T. T., and undefined 2021, “Using machine learning to examine street green space types at a high spatial resolution: Application in Los Angeles County on socioeconomic disparities in,” Elsevier.
L. Dai, C. Zheng, Z. Dong, Y. Yao, … R. W.-C. and E., and undefined 2021, “Analyzing the correlation between visual space and residents’ psychology in Wuhan, China using street-view images and deep-learning technique,” Elsevier.
H. Zhang, B. Chen, Z. Sun, Z. B.-U. F. & U. Greening, and undefined 2013, “Landscape perception and recreation needs in urban green space in Fuyang, Hangzhou, China,” Elsevier.
N. Kulkarni, “Color Thresholding Method for Image Segmentation of Natural Images,” International Journal of Image, Graphics and Signal Processing, vol. 4, no. 1, pp. 28–34, 2012, doi: 10.5815/ijigsp.2012.01.04.
J. M. Alvarez, T. Gevers, Y. LeCun, and A. M. Lopez, “Road scene segmentation from a single image,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 7578 LNCS, no. PART 7, pp. 376–389, 2012, doi: 10.1007/978-3-642-33786-4_28.
P. Sadeghi-Tehran, N. Virlet, K. Sabermanesh, and M. J. Hawkesford, “Multi-feature machine learning model for automatic segmentation of green fractional vegetation cover for high-throughput field phenotyping,” Plant Methods, vol. 13, no. 1, Nov. 2017, doi: 10.1186/S13007-017-0253-8.
R. Zhang, S. Candra, K. V.-2015 I. international, and undefined 2015, “Sensor fusion for semantic segmentation of urban scenes,” ieeexplore.ieee.org.
G. J. Brostow, J. Shotton, J. Fauqueur, and R. Cipolla, “Segmentation and recognition using structure from motion point clouds,” Lecture Notes in Computer Science, vol. 5302 LNCS, no. PART 1, pp. 44–57, 2008, doi: 10.1007/978-3-540-88682-2_5.
T. Rehman, M. Mahmud, Y. Chang, … J. J.-… and electronics in, and undefined 2019, “Current and future applications of statistical machine learning algorithms for agricultural machine vision systems,” Elsevier.
D. J. Bora, A. K. Gupta, and F. A. Khan, “Comparing the Performance of L*A*B* and HSV Color Spaces with Respect to Color Image Segmentation,” Jun. 2015.
S. Khan et al., “Human Action Recognition: A Paradigm of Best Deep Learning Features Selection and Serial Based Extended Fusion,” mdpi.com, 2021, doi: 10.3390/s21237941.
J. Taipalmaa, … N. P.-2020 I. international, and undefined 2020, “Different color spaces in deep learning-based water segmentation for autonomous marine operations,” ieeexplore.ieee.org.
Dr. S. Manoharan, “Performance Analysis of Clustering Based Image Segmentation Techniques,” Journal of Innovative Image Processing, vol. 2, no. 1, pp. 14–24, 2020, doi: 10.36548/jiip.2020.1.002.
R. Yogamangalam and B. Karthikeyan, “Segmentation techniques comparison in image processing,” International Journal of Eng. and Technology, vol. 5, no. 1, pp. 307–313, 2013.
N. Sharma et al., “Automated medical image segmentation techniques,” Journal of Medical Physics, vol. 35, no. 1. pp. 3–14, 2010. doi: 10.4103/0971-6203.58777.
H. J. He, C. Zheng, and D. W. Sun, “Image Segmentation Techniques,” in Computer Vision Technology for Food Quality Evaluation: Second Edition, 2016, pp. 45–63. doi: 10.1016/B978-0-12-802232-0.00002-5.
E. Akbal, “An automated environmental sound classification methods based on statistical and textural feature,” Applied Acoustics, vol. 167, 2020, doi: 10.1016/j.apacoust.2020.107413.
D. R. Patel, V. Vakharia, and M. B. Kiran, “Texture classification of machined surfaces using image processing and machine learning techniques,” FME Transactions, vol. 47, no. 4, pp. 865–872, 2019, doi: 10.5937/fmet1904865P.
M. R. Rudnick, S. Goldfarb, and J. Tumlin, “Contrast-induced nephropathy: Is the picture any clearer?,” Clinical Journal of the American Society of Nephrology, vol. 3, no. 1, pp. 261–262, 2008, doi: 10.2215/CJN.04951107.
K. Hasikin and N. A. Mat Isa, “Adaptive fuzzy contrast factor enhancement technique for low contrast and nonuniform illumination images,” Signal Image Video Process, vol. 8, no. 8, pp. 1591–1603, 2014, doi: 10.1007/s11760-012-0398-x.
A. Roche, G. Malandain, X. Pennec, and N. Ayache, “The correlation ratio as a new similarity measure for multimodal image registration,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 1998, vol. 1496, pp. 1115–1124. doi: 10.1007/bfb0056301.
M. Van Heel, “Similarity measures between images,” Ultramicroscopy, vol. 21, no. 1, pp. 95–100, 1987, doi: 10.1016/0304-3991(87)90010-6.
T. Wei, J. Wei, K. Zhang, H. Zhao, and L. Zhang, “Grayscale image recording on Ge2Sb2Te5 thin films through laser-induced structural evolution,” Sci Rep, vol. 7, 2017, doi: 10.1038/srep42712.
R. K. Mohammed and H. A. Abdullah, “Implementation of digital and analog modulation systems using FPGA,” Indonesian Journal of Electrical Engineering and Computer Science, vol. 18, no. 1, pp. 485–493, 2020, doi: 10.11591/ijeecs.v18.i1.pp485-493.
Y. Asaoka et al., “29.1: Polarizer-Free Reflective LCD Combined with Ultra Low-Power Driving Technology,” SID Symposium Digest of Technical Papers, vol. 40, no. 1, p. 395, 2009, doi: 10.1889/1.3256797.
E. Y. Lin, A. Rahmawati, J. H. Ko, and J. C. Liu, “Extraction of yttrium and europium from waste cathode-ray tube (CRT) phosphor by subcritical water,” Sep Purif Technol, vol. 192, pp. 166–175, 2018, doi: 10.1016/j.seppur.2017.10.004.
S. Hwang, Y. Cheon, S. Han, I. Jang, and J. W. Shin, “Enhancement of Coded Speech Using Neural Network-Based Side Information,” IEEE Access, vol. 9, pp. 121532–121540, 2021, doi: 10.1109/ACCESS.2021.3108784.
E. A. Boonstra and H. A. Slagter, “The Dialectics of Free Energy Minimization,” Front Syst Neurosci, vol. 13, 2019, doi: 10.3389/fnsys.2019.00042.
A. Frost, “Overvaluing uniformity,” Virginia Law Review, vol. 94, no. 7. pp. 1567–1639, 2008.
S. Van Der Walt et al., “Scikit-image: Image processing in python,” PeerJ, vol. 2014, no. 1, 2014, doi: 10.7717/peerj.453.
M. Tixier, C. Cian, P. A. Barraud, R. Laboissiere, and S. Rousset, “The interaction between long-term memory and postural control: Different effects of episodic and semantic tasks,” Motor Control, vol. 25, no. 2, pp. 182–197, 2021, doi: 10.1123/MC.2020-0061.
G. Singh and M. A. Ansari, “Efficient detection of brain tumor from MRIs using K-means segmentation and normalized histogram,” in India International Conference on Information Processing, IICIP 2016 - Proceedings, 2017. doi: 10.1109/IICIP.2016.7975365.
Y. Kinoshita and H. Kiya, “Hue-correction scheme based on constant-hue plane for deep-learning-based color-image enhancement,” IEEE Access, vol. 8, pp. 9540–9550, 2020, doi: 10.1109/ACCESS.2020.2964823.
M. J. Pearson et al., “Evidence of Intrinsic Impairment of Osteoblast Phenotype at the Curve Apex in Girls With Adolescent Idiopathic Scoliosis,” Spine Deform, vol. 7, no. 4, pp. 533–542, 2019, doi: 10.1016/j.jspd.2018.11.016.
S. Kunz, S. Haasova, and A. Florack, “Fifty shades of food: The influence of package color saturation on health and taste in consumer judgments,” Psychol Mark, vol. 37, no. 7, pp. 900–912, 2020, doi: 10.1002/mar.21317.
J. K. Patil and R. Kumar, “Analysis of content based image retrieval for plant leaf diseases using color, shape and texture features,” Engineering in Agriculture, Environment and Food, vol. 10, no. 2, 2017, doi: 10.1016/j.eaef.2016.11.004.
L. Y. Hanger, A. Lotz, and S. Lepeniotis, “Descriptive profiles of selected high intensity sweeteners (HIS), HIS blends, and sucrose,” J Food Sci, vol. 61, no. 2, pp. 456–459, 1996, doi: 10.1111/j.1365-2621.1996.tb14216.x.
M. Domino et al., “Selection of Image Texture Analysis and Color Model in the Advanced Image Processing of Thermal Images of Horses following Exercise,” Animals, vol. 12, no. 4, 2022, doi: 10.3390/ani12040444.
S. Pearl and A. Dorothy, “Converting RGB to HSV,” Mechatronics, 2022.
B. Robot, “Hsv_Writeup,” Mechatronics, 2005.
P. M. Nishad and R. Manicka Chezian, “Various Colour Spaces and Colour Space Conversion Algorithms,” Journal of Global Research in Computer Science, vol. 4, no. 1, pp. 44–48, 2013.
L. B. Wexner, “The degree to which colors (hues) are associated with mood-tones,” Journal of Applied Psychology, vol. 38, no. 6, pp. 432–435, 1954, doi: 10.1037/h0062181.
M. M. Hennink, B. N. Kaiser, and M. B. Weber, “What Influences Saturation? Estimating Sample Sizes in Focus Group Research,” Qual Health Res, vol. 29, no. 10, pp. 1483–1496, 2019, doi: 10.1177/1049732318821692.
R. R. Burton and J. S. Brown, “An investigation of computer coaching for informal learning activities,” Int J Man Mach Stud, vol. 11, no. 1, pp. 5–24, 1979, doi: 10.1016/S0020-7373(79)80003-6.
S. N. Utami, B. Kiswanjaya, S. I. Syahraini, and P. Ustriyana, “Tolerance limit value of brightness and contrast adjustment on digitized radiographs,” in Journal of Physics: Conference Series, 2017, vol. 884, no. 1. doi: 10.1088/1742-6596/884/1/012052.
B. Voisiat, W. Wang, M. Holzhey, and A. F. Lasagni, “Improving the homogeneity of diffraction based colours by fabricating periodic patterns with gradient spatial period using Direct Laser Interference Patterning,” Sci Rep, vol. 9, no. 1, 2019, doi: 10.1038/s41598-019-44212-4.
I. Kurniastuti, E. N. I. Yuliati, F. Yudianto, and T. D. Wulan, “Determination of Hue Saturation Value (HSV) color feature in kidney histology image,” in Journal of Physics: Conference Series, 2022, vol. 2157, no. 1. doi: 10.1088/1742-6596/2157/1/012020.
L. L. P. Nguyen, L. Baranyai, D. Nagy, P. V. Mahajan, V. Zsom-Muha, and T. Zsom, “Color analysis of horticultural produces using hue spectra fingerprinting,” MethodsX, vol. 8, 2021, doi: 10.1016/j.mex.2021.101594.
Y. P. Huang, T. H. Wang, and H. Basanta, “Using Fuzzy Mask R-CNN Model to Automatically Identify Tomato Ripeness,” IEEE Access, vol. 8, pp. 207672–207682, 2020, doi: 10.1109/ACCESS.2020.3038184.
C. E. Widodo, K. Adi, and R. Gernowo, “Medical image processing using python and open cv,” in Journal of Physics: Conference Series, 2020, vol. 1524, no. 1. doi: 10.1088/1742-6596/1524/1/012003.
A. Bhatnagar, R. Patel, M. Gupta, M. Pal, and L. Kumar, “Customized sorting and packaging machine,” Telkomnika (Telecommunication Computing Electronics and Control), vol. 19, no. 4, pp. 1326–1333, 2021, doi: 10.12928/TELKOMNIKA.v19i4.16786.
X. Wang, C. Song, C. Yang, and Y. Xie, “Process working condition recognition based on the fusion of morphological and pixel set features of froth for froth flotation,” Miner Eng, vol. 128, pp. 17–26, 2018, doi: 10.1016/j.mineng.2018.08.017.
“Linda G. Shapiro and George C. Stockman (2001):... - Google Scholar.”
L. Barghout, 543 L Lee - US Patent App. 10/618, and undefined 2004, “Perceptual information processing system,” Google Patents, no. 10.
S. Minaee, Y. Boykov, F. Porikli, A. Plaza, N. Kehtarnavaz, and D. Terzopoulos, “Image Segmentation Using Deep Learning: A Survey,” IEEE Trans Pattern Anal Mach Intell, vol. 44, no. 7, pp. 3523–3542, 2022, doi: 10.1109/TPAMI.2021.3059968.
W. Weng and X. Zhu, “INet: Convolutional Networks for Biomedical Image Segmentation,” IEEE Access, vol. 9, pp. 16591–16603, 2021, doi: 10.1109/ACCESS.2021.3053408.
S. Khan et al., “Human Action Recognition: A Paradigm of Best Deep Learning Features Selection and Serial Based Extended Fusion,” mdpi.com, 2021, doi: 10.3390/s21237941.
A. Hafiz, G. B.-I. journal of multimedia information, and undefined 2020, “A survey on instance segmentation: state of the art,” Springer.
Y. Yi, M. L.-P. Recognition, and undefined 2016, “Human action recognition with graph-based multiple-instance learning,” Elsevier.
S. Liu, L. Qi, H. Qin, J. Shi, J. J.-P. of the IEEE, and undefined 2018, “Path aggregation network for instance segmentation,” openaccess.thecvf.com.
B. Romera-Paredes and P. H. S. Torr, “Recurrent instance segmentation,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9910, pp. 312–329, 2016, doi: 10.1007/978-3-319-46466-4_19.
N. Gao, Y. Shan, Y. Wang, X. Zhao, and K. Huang, “SSAP: Single-Shot Instance Segmentation with Affinity Pyramid,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 2, pp. 661–673, 2021, doi: 10.1109/TCSVT.2020.2985420.
R. Fan, M. M. Cheng, Q. Hou, T. J. Mu, J. Wang, and S. M. Hu, “S4Net: Single stage salient-instance segmentation,” Comput Vis Media (Beijing), vol. 6, no. 2, pp. 191–204, 2020, doi: 10.1007/s41095-020-0173-9.
D. Bolya, C. Zhou, F. Xiao, and Y. J. Lee, “YOLACT++ Better Real-Time Instance Segmentation,” IEEE Trans Pattern Anal Mach Intell, vol. 44, no. 2, pp. 1108–1121, 2022, doi: 10.1109/TPAMI.2020.3014297.
I. Papadeas, L. Tsochatzidis, A. Amanatiadis, and I. Pratikakis, “Real-time semantic image segmentation with deep learning for autonomous driving: A survey,” Applied Sciences (Switzerland), vol. 11, no. 19, 2021, doi: 10.3390/app11198802.
S. Asgari Taghanaki, K. Abhishek, J. P. Cohen, J. Cohen-Adad, and G. Hamarneh, “Deep semantic segmentation of natural and medical images: a review,” Artif Intell Rev, vol. 54, no. 1, pp. 137–178, 2021, doi: 10.1007/s10462-020-09854-1.
X. Liu, Z. Deng, and Y. Yang, “Recent progress in semantic image segmentation,” Artif Intell Rev, vol. 52, no. 2, pp. 1089–1106, 2019, doi: 10.1007/s10462-018-9641-3.
O. L. F. de Carvalho et al., “Panoptic Segmentation Meets Remote Sensing,” Remote Sens (Basel), vol. 14, no. 4, 2022, doi: 10.3390/rs14040965.
Q. Chen, A. Cheng, X. He, P. Wang, and J. Cheng, “SpatialFlow: Bridging All Tasks for Panoptic Segmentation,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 6, pp. 2288–2300, 2021, doi: 10.1109/TCSVT.2020.3020257.
R. Mohan and A. Valada, “EfficientPS: Efficient Panoptic Segmentation,” Int J Comput Vis, vol. 129, no. 5, pp. 1551–1579, 2021, doi: 10.1007/s11263-021-01445-z.
H. Gouk, E. Frank, B. Pfahringer, and M. J. Cree, “Regularisation of neural networks by enforcing Lipschitz continuity,” Mach Learn, vol. 110, no. 2, pp. 393–416, 2021, doi: 10.1007/s10994-020-05929-w.
Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and P. S. Yu, “A Comprehensive Survey on Graph Neural Networks,” IEEE Trans Neural Netw Learn Syst, vol. 32, no. 1, pp. 4–24, 2021, doi: 10.1109/TNNLS.2020.2978386.
Y. V. R. Nagapawan, K. B. Prakash, and G. R. Kanagachidambaresan, “Convolutional Neural Network,” in EAI/Springer Innovations in Communication and Computing, 2021, pp. 45–51. doi: 10.1007/978-3-030-57077-4_6.
S. Lyu and J. Liu, “Convolutional recurrent neural networks for text classification,” Journal of Database Management, vol. 32, no. 4, pp. 65–82, 2021, doi: 10.4018/JDM.2021100105.
Sarang Narkhede, “Understanding AUC - ROC Curve,” Towards Data Science, pp. 6–11, 2019.
M. Latah and L. Toker, “Minimizing false positive rate for DoS attack detection: A hybrid SDN-based approach,” ICT Express, vol. 6, no. 2, pp. 125–127, 2020, doi: 10.1016/j.icte.2019.11.002.
C. S. Hong and T. G. Oh, “TPR-TNR plot for confusion matrix,” Commun Stat Appl Methods, vol. 28, no. 2, pp. 161–169, 2021, doi: 10.29220/CSAM.2021.28.2.161.
J. N. Jannath and M. S. B. S, “Detection of malicious Android applications using Ontology-based intelligent model in mobile cloud environment,” Journal of Information Security and Applications, vol. 58, 2021, doi: 10.1016/j.jisa.2021.102751.
X. Zeng, Y. Zhang, X. Wang, K. Chen, D. Li, and W. Yang, “Fine-Grained Image Retrieval via Piecewise Cross Entropy loss,” Image Vis Comput, vol. 93, 2020, doi: 10.1016/j.imavis.2019.10.006.
J. Lu and S. Steinerberger, “Neural collapse under cross-entropy loss,” Appl Comput Harmon Anal, vol. 59, pp. 224–241, 2022, doi: 10.1016/j.acha.2021.12.011.
X. Zeng, Y. Zhang, X. Wang, K. Chen, D. Li, and W. Yang, “Fine-Grained Image Retrieval via Piecewise Cross Entropy loss,” Image Vis Comput, vol. 93, 2020, doi: 10.1016/j.imavis.2019.10.006.
M. Martinez and R. Stiefelhagen, “Taming the Cross Entropy Loss,” in Lecture Notes in Computer Science, 2019, vol. 11269 LNCS, pp. 628–637. doi: 10.1007/978-3-030-12939-2_43.
M. Hasnain, M. F. Pasha, I. Ghani, M. Imran, M. Y. Alzahrani, and R. Budiarto, “Evaluating Trust Prediction and Confusion Matrix Measures for Web Services Ranking,” IEEE Access, vol. 8, pp. 90847–90861, 2020, doi: 10.1109/ACCESS.2020.2994222.
J. Xu, Y. Zhang, and D. Miao, “Three-way confusion matrix for classification: A measure driven view,” Inf Sci (N Y), vol. 507, pp. 772–794, 2020, doi: 10.1016/j.ins.2019.06.064.