Shaping the PoDIUM evaluation methodology

PoDIUM’s evaluation methodology is key to assessing the technical performance, user acceptance, and socio-economic impact of the project’s CCAM (Cooperative, Connected, and Automated Mobility) solutions, and providing actionable insights that can guide the successful deployment of these solutions in real-world scenarios.

The evaluation methodology builds on methodologies from previous projects like ICT4CART and 5G-IANA, incorporating lessons learned to tailor it specifically to PoDIUM’s goals, and ensure that PoDIUM’s outcomes are reliable, reproducible, and comparable across different scenarios and environments.

The methodology is structured around a phased approach inspired by the FESTA V-model, which includes three main phases:

• Preparation: definition of the pilot’s functions, use cases, research questions, and KPIs (Key Performance Indicators), set-up of the study design, measures, and participant recruitment;
• Execution: data collection based on the predefined test cases;
• Analysis: processing and analysis of collected data to answer research questions, including impact assessments and deployment scenario analysis.

Central to this methodology are test cases organised into six test suites, each linked to specific PoDIUM platform functions or use cases. Pre-tests will be conducted to validate the experimental setups, followed by comprehensive data collection. The results are evaluated against target KPIs, and alternative approaches, such as simulations or network stress tests, are used when direct measurement isn’t feasible. The collected data is analysed to refine the evaluation model, with the results documented and integrated into recommendations for deploying CCAM solutions.

More information