Metamorphic Testing of an Automated Parking System: An Experience Report

Dave Towey, Zepei Luo, Ziqi Zheng, Peijian Zhou, Junbo Yang, Puttipatt Ingkasit, Changyang Lao, Matthew Pike, Yifan Zhang

COMPSAC 2023 - 47th Annual Computers, Software, and Applications Conference, August 2023

[DOI] [Cite]

Dave Towey, Zepei Luo, Ziqi Zheng, Peijian Zhou, Junbo Yang, Puttipatt Ingkasit, Changyang Lao, Matthew Pike, Yifan Zhang. 2023. Metamorphic Testing of an Automated Parking System: An Experience Report. In COMPSAC 2023 - 47th Annual Computers, Software, and Applications Conference. DOI:https://doi.org/10.1109/COMPSAC57700.2023.00274

Dave Towey and Zepei Luo and Ziqi Zheng and Peijian Zhou and Junbo Yang and Puttipatt Ingkasit and Changyang Lao and Matthew Pike and Yifan Zhang. (2023). Metamorphic Testing of an Automated Parking System: An Experience Report. COMPSAC 2023 - 47th Annual Computers, Software, and Applications Conference. https://doi.org/10.1109/COMPSAC57700.2023.00274

Dave Towey and Zepei Luo and Ziqi Zheng and Peijian Zhou and Junbo Yang and Puttipatt Ingkasit and Changyang Lao and Matthew Pike and Yifan Zhang. "Metamorphic Testing of an Automated Parking System: An Experience Report." COMPSAC 2023 - 47th Annual Computers, Software, and Applications Conference, 2023. https://doi.org/10.1109/COMPSAC57700.2023.00274

Dave Towey, Zepei Luo, Ziqi Zheng, Peijian Zhou, Junbo Yang, Puttipatt Ingkasit, Changyang Lao, Matthew Pike, Yifan Zhang. 2023. Metamorphic Testing of an Automated Parking System: An Experience Report. COMPSAC 2023 - 47th Annual Computers, Software, and Applications Conference. doi:10.1109/COMPSAC57700.2023.00274

Dave Towey and Zepei Luo and Ziqi Zheng and Peijian Zhou and Junbo Yang and Puttipatt Ingkasit and Changyang Lao and Matthew Pike and Yifan Zhang, "Metamorphic Testing of an Automated Parking System: An Experience Report," COMPSAC 2023 - 47th Annual Computers, Software, and Applications Conference, 2023. doi: 10.1109/COMPSAC57700.2023.00274

@inproceedings{compsac-2023-2,
  title={Metamorphic Testing of an Automated Parking System: An Experience Report},
  author={Dave Towey and Zepei Luo and Ziqi Zheng and Peijian Zhou and Junbo Yang and Puttipatt Ingkasit and Changyang Lao and Matthew Pike and Yifan Zhang},
  booktitle={COMPSAC 2023 - 47th Annual Computers, Software, and Applications Conference},
  year={2023},
  doi={10.1109/COMPSAC57700.2023.00274}
}

Metamorphic testing, Automated parking systems, Test oracle problem, Baidu Apollo, Mutation analysis, Software quality assurance, Automated driving systems

Abstract

Automated Driving Systems (ADSs) have gained popularity, but their reliability and safety remain key concerns due to the test oracle problem. This paper reports on the application of Metamorphic Testing (MT) to evaluate an Automated Parking System (APS) in Baidu Apollo, an open-source ADS platform. The study integrates Mutation Analysis (MA) to assess the sufficiency of MT-based testing. The experience highlights how MT can alleviate oracle-related challenges by defining Metamorphic Relations (MRs) to validate APS behavior. The study also explores the creation of Open Educational Resources (OERs) for disseminating knowledge on MT in ADS testing. The results demonstrate the effectiveness of MT in identifying potential issues in APS and underscore the value of MA in refining MR-based testing strategies. The paper concludes with recommendations for future improvements in MT applications for ADS evaluation.