Testing and evaluation methodology for AI-driven CCAM systems

Last modified on 18 July, 2025

AI has revolutionised Connected, Cooperative and Automated Mobility (CCAM) solutions by enabling AI models to be trained on vast amounts of data. However, AI remains underexplored in terms of explainability, privacy, ethics and accountability. The EC CCAM project AIthena aims to contribute to building Explainable AI (XAI) into CCAM development and testing frameworks by exploring three main AI pillars: data management, models, and testing.

The project has developed a human-centred methodology, including a set of Key Performance Indicators (KPI) for XAI, to derive trustworthy AI dimensions from user-identified group needs in CCAM applications.

Four use cases have been established to further dissect and understand how AI can be explained and understood throughout its decision-making process, which will be used to demonstrate the AIthena methodology: perception, situation awareness/understanding, decision making and traffic management. Each use case is an in-depth analysis of how different data sets are considered in the perception, understanding, decision-making and deployment of AI in vehicles:

• Use case 1: data cards, sensors and visualisation tools (as they gather the necessary information to detect pedestrians in the traffic environment);
• Use case 2: algorithms and perception data from the vehicle (to explain how AI makes its decisions);
• Use case 3: examines how AI explains its decisions to humans (drivers in the vehicle), to build trust in the technology;
• Use case 4: examines how AI-equipped vehicles are deployed in traffic environments and how these vehicles can impact or improve traffic dynamics in terms of comfort, efficiency and safety for drivers and road users.

All related information, which form the basis for benchmark scenarios and KPIs definition, can be found in the recent report documenting the AI and CCAM testing methodology used in the project.

Source: The original article can be found here