Metamorphic Test-Based Robustness Evaluation of Object Detection Models in Safety Critical Systems 


Vol. 50,  No. 7, pp. 995-1006, Jul.  2025
10.7840/kics.2025.50.7.995


PDF Full-Text
  Abstract

In recent years, artificial intelligence (AI) technology has made rapid progress and is widely used in various fields such as natural language processing and computer vision. However, the black-box nature of AI models raises reliability concerns, particularly in safety-critical systems (ScSs) such as autonomous vehicles (AVs) and medical systems, where understanding model behavior is crucial. This study proposes a robustness evaluation method for AV object detection models (ODMs) by integrating metamorphic testing and adversarial attack techniques. Using a YOLOv5-based ODM, the model was tested to detect essential objects in traffic safety scenarios, including pedestrians, traffic lights, and road signs. Various image augmentations simulating physical deformations (e.g., rotation, tilt) and weather conditions (e.g., snow, rain) were applied to measure model robustness. Results showed that model performance varied significantly with specific augmentations, revealing vulnerabilities. Notably, the pedestrian detection model, despite its high accuracy during training, showed only 35.8% robustness against augmented images. This highlights that even high-performing models in general conditions may fail under diverse environmental factors. These findings underscore the importance of rigorous robustness evaluation for AI models in ScSs to ensure reliability and safety in real-world applications like AVs.

  Statistics
Cumulative Counts from November, 2022
Multiple requests among the same browser session are counted as one view. If you mouse over a chart, the values of data points will be shown.


  Related Articles
  Cite this article

[IEEE Style]

J. Suh, J. Lee, J. Kim, T. Lee, "Metamorphic Test-Based Robustness Evaluation of Object Detection Models in Safety Critical Systems," The Journal of Korean Institute of Communications and Information Sciences, vol. 50, no. 7, pp. 995-1006, 2025. DOI: 10.7840/kics.2025.50.7.995.

[ACM Style]

Jun-Cheol Suh, Jung-Been Lee, Jeong-Dong Kim, and Taek Lee. 2025. Metamorphic Test-Based Robustness Evaluation of Object Detection Models in Safety Critical Systems. The Journal of Korean Institute of Communications and Information Sciences, 50, 7, (2025), 995-1006. DOI: 10.7840/kics.2025.50.7.995.

[KICS Style]

Jun-Cheol Suh, Jung-Been Lee, Jeong-Dong Kim, Taek Lee, "Metamorphic Test-Based Robustness Evaluation of Object Detection Models in Safety Critical Systems," The Journal of Korean Institute of Communications and Information Sciences, vol. 50, no. 7, pp. 995-1006, 7. 2025. (https://doi.org/10.7840/kics.2025.50.7.995)
Vol. 50, No. 7 Index

Search
(IN TITLE, AUTHOR, ABSTRACT,KEYWORDS)

Advanced Search ▼

POPULAR KEYWORDS
(TOP 10 KEYWORDS)

Recent Publications
(LAST 3 YEARS)






pISSN : 1226 - 4717
eISSN : 2287 - 3880