Guidelines for reporting evaluation outcomes and EU-CEM feedback

Requirements for reporting the evaluation methodology and its outcomes can vary depending on the study commissioner. Evaluation reports are usually published at the end of the project when the evaluation is completed. However, they can also be released during the project, such as through a separate evaluation plan.

Effective reporting addresses the distinct information needs of different audiences. Given these diverse audiences, it is recommended to produce multiple report formats, ranging from highly detailed technical documents to executive summaries and presentations. Particular attention should be given to effectively communicating information to stakeholders without technical, scientific, or evaluation-related backgrounds.

Reports should clearly document the assumptions, methodological steps, and limitations of the evaluation process. Comprehensive reporting clarifies how outcomes and conclusions were reached, how the approach can be enhanced, and what research gaps remain.

The evaluation report should answer all research questions and consider any hypotheses. If some research questions remain unanswered, the report should explain the reason for this. Answers to research questions and related indicator values should be structured according to the research question in the evaluation plan. A summary table can provide an overview of the research questions and their answers.

Besides presenting results, providing context and interpretation is essential, as results alone are often insufficient for fully understanding the underlying phenomena. For example, it is crucial to communicate whether observed impacts on traffic flow efficiency resulted from shorter headways or differences in speed. A thorough understanding of causality helps users of the results maximise the benefits and minimise the drawbacks of CCAM.

Conclusions can be drawn at all evaluation levels and in an overarching manner, not forgetting insights on the impacts of CCAM on sustainability. In addition, it is important to list topics considered important for future research on the most significant aspects that lack understanding or knowledge.

The EU-CEM Handbook is designed to be enhanced over time through insights, lessons learned, and feedback from the CCAM community. Therefore, feedback on EU-CEM is appreciated to benefit future projects. A form is available for that purpose in the Knowledge Base on Connected and Automated Driving (CAD). Additionally, the EU-CEM developer team can be contacted at info@connectedautomateddriving.eu.

For deriving this feedback, it is recommended that at the end of the evaluation, the evaluation team reflects on the evaluation process. Consider questions such as:

• What went well?
• What did not, and why?
• Was EU-CEM helpful in the process? How could it be improved?

The EU-CEM developer team is interested in hearing: 

• Was EU-CEM followed, what were the deviations and the reasons for them? 
• Which chapters of EU-CEM provided useful guidelines? In what way, and where, were they not applicable or useful?
• What new best practices did the project successfully implement, and which ones would you recommend to others? 
• What new or updated guidelines and impact areas would be useful in future updates of the EU-CEM? Were all relevant use-case perspectives covered? Were there other gaps identified in the EU-CEM?