Lessons Learned Overview

Type of lesson learnedTopic of lesson learnedProject nameR+I ClusterDeliverableBrief summary of the lesson learned
FinancialVenture funding for AI startupsAI4EUR+I 5.4 AI for situational awarenessD9.1 The AI Vision for Europe• Difficulty of scaling start-ups: Lack of venture funding at later stages has been identified as an issue, much lower than in other continents.
• European start-ups experience difficulty selling their products/services in Europe compared to North American and Asian markets. Furthermore, most start-ups are acquired by US and Asian players at early stages.
FinancialData re-use can bring additional resultsCARTRER+I 7.2 Common Evaluation MethodologyD4.1 Data exchange platform for automation pilots including a sharing strategy (Chapter 5, pg. 24) • Data collected in past projects should be re-used, as the projects that the data are principally collected for can only cover a small portion of all possible research the data could be used for. Large European FOT/NDS projects typically spend 10% of project time and cost on performing analyses, with the rest spent on testing and data collection.
• With additional funding of 10% of the original project cost, project results could even be doubled.
FinancialA more attractive business model is needed for cities and operatorsFABULOSR+I 1.2 Field Operational Trials (FOTs)D5.6 Future application and impact of project results and learnings• The business model is still unclear, and does not make the system attractive. The current technology does not allow to build a clear vision of the cost/revenue expected, neither on the impact on jobs.
FinancialEvolution of procurement policies is requiredFABULOSR+I 1.2 Field Operational Trials (FOTs)D5.6 Future application and impact of project results and learnings• Need to create a innovation-friendly procurement market.
• Foster a better balance between detailed regulation on autonomous vehicles and a possibility of "learning by doing".
• Leverage innovation procurement through modernised framewords, and direct porcurement investment.
FinancialProcurement and availability of automation-ready vehicle platformsMuCCAR+I 1.1 Environment perception technologies for CCAMMulti-Car Collision Avoidance Report• Timely access to a suitable automation-ready vehicle platform has been challenging in the project.
• Ready to buy solutions are expensive and developing customized solutions within the project requires a lot of resources.
• Early access to a physical platform during development has proven convenient although simulation has helped on this side to bench test algorithms and part of the hardware.
FinancialBarriers for CCAM business modelsSHOWR+I 6.2 Socio-economic and environmental impact analysis and target-based assessment of CCAM benefitsD2.1: Benchmarking of existing business / operating models + best practices• CAPEX and OPEX are the main barriers for earning money, but OPEX could offer business chances by extending the value chain for the mobility service itself by opening it to new participants.
• Parking vehicles and therefore not used services do not earn money, so it is quite important to maximize the utilization. So considering a mixed mobility services approach (MaaS, LaaS and DRT) could open the way to new business models or to extend existing established business.
• Consideration of the whole business ecosystem including all sub-systems (analysing the second and third line within the mobility service) and all user and operating roles is very important to generate an adequate view with all chances, costs and revenue streams to get a complete picture and to identify business potentials for cost reduction or increased business success
Financial
Organisational/Operational
Twinning activity EU/US could be further enhanced interACT • Twinning with the AVIntent project went well with the academic partners of the projects. Even better if financial resources would have been planned and allocated for both projects for exchange meeting and conference participation.
• Further, the matching was good with regards to the objectives of the projects. However, the project sizes could be better balanced (only one academic partner from US project/no industrial partners; project budget and runtime) and guidance for organizational/legal issues from the EU/US would be helpful.
Legal + regulationsGenerate a common legal glossary starting pointAdaptIVe (FP7 project)D2.1 System classification and glossary • Due to the complexity in the legal domain and the required precision, it was important to develop a good starting point and a glossary for the general discussion with experts from different fields. This was quite a challenge, and it was very appropriate to consider this task from the beginning of the project while defining timelines and plans.
Legal + regulationsData legal requirements: Cloud LSVAR+I 5.5 System architecture for data sharingD2.6 Final report on data legal requirements and
implemented data protection approaches
In this project, an extensive study on legal requirements of personal and non-personal data was conducted. Main findings include:
• All contributors to AD(A)S/cloud technology across the supply chain to the ultimate vehicle will likely be exposed to liability based on being either data processors or data controllers.
• There is a gap between the implementation of the GDPR and its application to Cloud based systems. The use of industry Codes can be of significant assistance in that regard.
• The only way of managing this liability risk in data management is to ensure processes conform with the over-arching provisions of the GDPR from acquisition to destruction. At each stage of the development of the technology, the privacy impact consideration needs to be addressed.
Legal + regulationsImportance of taking into account AD for future legislative and standardisation: CoEXISTR+I 6.1 Societal, citizen and user needs - for needs-based CCAM solution development and deploymentD4.6 Automation-ready Action Plan for each road authority•Automated driving and its recognisable effects must be taken into account today in future legislative and standardisation procedures.
•This applies, for example, to the planned liberalisation of the Passenger Transport Act from the perspective of a strong and efficient Public Transport in terms of services of general interest.
Legal + regulationsPolitical learningsFABULOSD5.6 Future application and impact of project results and learnings• Evolution of procurement policies required : create a more innovation-friendly procurement market, allowing a better balance between detailed regulation on autonomous vehicles and a possibility of “learning by doing”.
• Leverage innovation procurement through modernised frameworks, and direct procurement investment.
Legal + regulationsRegulation harmonisation to allow deploymentFABULOSR+I 1.2 Field Operational Trials (FOTs)D5.6 Future application and impact of project results and learnings• The commercialsation and implementation of automated fleets in the transport system require the improvement of several legislative and technical features.
• More harmonized EU-wide regulation is needed to enable the implementation of pilots and integration of automated transportation in cities.
Legal + regulationsRegulation for external HMI interACT D4.2 Final human-vehicle interaction strategies for the interACT Avs• Regulation on national, European and international level for external HMI components and indication for Automated Vehicles would be needed to allow easy-to-learn, comprehensible information transfer to other road users, interACT results were shared with the relevant regulation bodies but future projects should further contribute to this discussion as many research questions and details are not yet explored.
Legal + regulationsEnsuring confidentiality of collected informationL3PilotR+I 1.1 PilotsD3.4 Evaluation plan
Chapter 6 Lessons Learned pp. 103–104
• Evaluation details concerning the driving dynamics of ADFs are sensitive to vehicle manufacturers. Therefore, it is vital to keep vehicle telemetry data collected during piloting confidential. Additionally, any data made public must not be such that it can be used to rank and compare manufacturers, as this could have negative impact on OEM competitiveness.
To address this:
• Distribution of commercially sensitive data is restricted by limiting the handling of particularly detailed data to one or two research partners.
• Using an agreed-upon common data format, methodology and analysis toolkit, which can be used by different analysts at different pilot sites and when investigating different ADFs.
• A data-sharing process which pseudonymises data at a suitable level of aggregation from each pilot site is used to ensure a suitable level of anonymity. In addition, results from more than one pilot site are merged in a way that protects sensitive information.
The method facilitates the provision of insights concerning ADFs without violating privacy and is enabled by the common data format, methodology and analysis toolkit. However, the method requires that any outcomes for an ADF presented publically have to have been piloted at more than one site.
Legal + regulationsSafety concepts when piloting prototype ADFs L3PilotR+I 1.1 PilotsD3.4 Evaluation plan
Chapter 6 Lessons Learned p. 102
• As the piloted ADFs are largely prototypes, special safety measures had to be taken during testing. This meant that ordinary members of the public could not be recruited to participate in testing. However, with the aid of several “safety concepts”, these restrictions could be overcome to achieve a more representative sample of drivers. Cconcepts deployed in L3Pilot included:
• Installing pedals on the passenger side of cars, facilitating interventions by a trained observer if needed.
• Providing an impression of the tested ADF by having an ordinary driver observe a trained driver’s interaction with the ADF from the passenger seat.
• A safety driver following the ordinary driver.
Legal + regulationsProject management SECREDAS R+I 5.1 Cybersecure components and systems• In the context of this type of project, having a dedicated work package on standardization is highly recommended, as this also generates insights and understanding for many partners as they develop their technologies.
Legal + regulationsNeed for a common legislation framework for vehicle useSHOWR+I 1.1 PilotsD3.1 Analysis report on legal, regulatory, institutional frameworks* It does not exist a common legislation regulating the vehicle use at European scale and every state has its own national legislation with regards to this topic.
* The creation of a common legislative framework in this matter could be interesting from the point of view of the vehicle final user.
* In this way, tasks like the obtention of a vehicle license exemption or the vehicle registration, would be equally obtained in every member state.
Legal + regulationsNeed for binding international standards and regulations (legislation) for automated
mobility
SHOWR+I 1.1 PilotsD3.1 Analysis report on legal, regulatory, institutional frameworks* As for today, cooperations and organizations are mainly working within their established fields of responsibility (mainly vehicle technology, behaviour in traffic).
• Even the need for adaption of existing international decision organizations and procedures has been widely recognized, it seems to be very difficult to change established organization forms or to introduce new fields of responsibilities into existing organizations.
• The need for change implies a suitable organization that can develop and implement the necessary changes and legal novation’s (maybe, completely new legislations are necessary, primarily focusing of the division of liability between technology and human behaviour). For instance, international cooperations, which are able to develop and realize binding international standards and regulations (legislation) for automated mobility.
Legal + regulationsDesign of regulations to avoid unsafe driving behaviour SIMUSAFER+I 6.3 Workforce development and knowledge enhancementSIMUSAFE infographic• Behavioural knowledge of road users can be also helpful to create regulations towards the reduction of fatal, serious and minor crashes through mitigation of unsafe user behaviour patterns.
Legal + regulationsLegal aspects to be addressed to facilitate future measuresTransAIDR+I 4.1 Physical and Digital Infrastructure (PDI)
R+I 4.2 Connectivity and Cooperative Systems
R+I 4.3 Fleet and traffic management in a CCAM eco-system
D8.2 Results of the meta-analysis (Chapter 4 lessons learned) • Although vehicle automation is at its start, the negative impacts of transition areas can only be avoided when specific countermeasures are initiated already now:
•This especially includes legal aspects like the definition of special signage for automated vehicles and their handling, as those aspects will take time.
•This also means signage at the roadside, including VMS content, and signage from automated vehicles to surrounding traffic.
Legal + regulationsEthics, regulation and standardisation vi-DASR+I 6.1 Societal, citizen and user needs - for needs-based CCAM solution development and deployment• There is a complex relationship between ethical and legal considerations, and each kind of viewpoint and social guiding principle has its limitations.
• To be aware of this may help developers to define issues where ethical reflection is highly important as law cannot address them adequately.
Organisational/OperationalImpact on standardisation and relevant bodies 5G CARMENR+I 6.3 Workforce development and knowledge enhancementD7.2 5G-CARMEN First dissemination, impact assessment and exploitation report • Towards a standardization process, one should monitor available standards in the telecommunication and in the automotive sector.
• The 5G-CARMEN use cases are used to derive 5G features for V2X, interworking with LTE V2X and C-ITS and other aspects.
• The pre-deployment trials identify missing elements in existing standards, proposing solutions to fill in the gaps. The final results of 5G-CARMEN will be used to recommend standardization activities and development roadmaps.
Organisational/OperationalTest Case design process. 5G CARMENR+I 4.2 Connectivity and Cooperative Systems
R+I 7.3 Test Data Exchange Framework
D5.1 5G-CARMEN Pilot Plan The following aspects need to be taken into account:
• Target KPI. Each test case targets a single KPI. The definition of the main KPI declares at least the reference points from which the measurement(s) will be performed.
• Complementary measurements. A secondary list of KPIs useful to interpret the values of the target KPI. Getting these measurements is not mandatory for the test case but, allows providing an additional set of results useful for analysis and interpretation of the relation between different KPIs.
• Pre-conditions. A precise description of the initial state of the system under test, required to start executing a test case sequence.
• Test case sequence. It specializes the set of processes needed for executing the experiment in the selected use case
• Methodology, calculation process and expected output. The acceptable values for variables that affect the testing procedure should be provided in advance. Other important aspects to take into consideration are the monitoring time, the iterations required, etc. The units that shall be used in the measurements are also defined here.
• Applicability. A list of features and capabilities which are required by the system in order to guarantee the feasibility of the test.
Organisational/OperationalCost/Benefit analysis5G CROCOR+I 6.2 Socio-economic and environmental impact analysis and target-based assessment of CCAM benefits5GCroCo_D5_2 Cost-Benefit Validation of Relevant 5GCroCo Business Potentials• A cost/benefit analysis identifies key directions in which the technology will certainly have an impact.
• The connected vehicle market, safe mobility, and the efforts done towards clean mobility are key aspects to derive estimated benefits of the full deployment of CCAM ecosystems.
• As the business is still immature, there are regulatory, technological and market creation and development uncertainties that need to be addressed from multiple perspectives, especially regulatory and market definition, in the years to come.
Organisational/OperationalFrequency allocation5G DRIVER+I 4.1 Physical and Digital Infrastructure (PDI)D2.2 5G-DRIVE Joint Architecture, Use Cases and Spectrum Plan• Frequency allocation is a competence of national authorities, so the radio frequency plan for eMBB and V2X in each region can easily differ.
• A compare-and-contrast approach has been used both for architecture and the radio frequency situation for eMBB and V2X in the EU and China.
Organisational/OperationalProcurement of cutting-edge equipment 5G MOBIXR+I 4.2 Connectivity and Cooperative Systems• Referring to 5G modems/chipsets, this lesson learned refers to the hardships faced in acquiring advanced equipment i.e., implementing very recently standardized functionality.
• This includes delays in manufacturing and shipping, but also stability and compatibility issues. Sufficient care must be taken in what concerns scheduling of activities, to avoid cascading delays.
Organisational/OperationalAI training data is in separate silosAI4EUR+I 5.4 AI for situational awarenessD9.1 The AI Vision for Europe• Difficulties accessing data and knowledge sources: data is often in silos, with no standardised data structures.
Organisational/OperationalStakeholders and players engagement C MOBILER+I 7.1 EU-wide knowledge base and stakeholder forum on CCAM• All the actors must be onboard to guarantee the whole data chain (from data source to the C-ITS service enjoyed by the user), i.e. data providers, public authorities, road operators, service providers and users must be engaged from the very beginning.
• User awareness must be increased. Efforts must be dedicated in the following years.
Organisational/OperationalStandardisation and interoperability for large-scale deployment C MOBILER+I 1.2 Field Operational Trials (FOTs)D3.1 Reference Architecture• Data and communications must be standardized to ensure interoperability and real deployment.
Organisational/OperationalCyber-security and automotive value chainCARAMELR+I 5.1 Cybersecure components and systems
R+I 6.1 Societal, citizen and user needs - for needs-based CCAM solution development and deployment
CARAMEL – D2.1 Report on Detailed Specification of Use Cases• When analyzing cybersecurity, one should consider the needs of the entire cyber-security and automotive value chains, ranging from: (a) the general public that uses digital communications and future automotive products, (b) the cyber-security solution providers, AI and ML methods developers, etc., (c) the infrastructure providers, represented by telecommunication infrastructure providers (telecom operators, ISPs), cloud service providers, and organisations with small, medium and large scale infrastructures that require a low-cost cyber-security investment, (d) vehicle manufacturing industry, i.e. automotive companies, equipment, system and solution providers for automotive industry, etc.
• CARAMEL further considers the needs of policy makers in EU + Member States for informed decisions regarding the security of modern infrastructures for the future vehicle industry.
• Additional benefits are considered in the case for standardisation, other special interest groups, open-source communities and researchers/academics.
Organisational/OperationalAgile and repetitive evaluation processesCARTRER+I 7.2 Common Evaluation MethodologyD4.2 Updated FESTA handbook, including a short version for automation FOTs and inventory of novel methods • We have to consider whether new approaches to evaluation are needed, such as a more agile approach, as is used in software development.
• A large FOT may take years to set-up and be conducted, especially, if everything (test sites, data collection, research questions, etc.) is built from a scratch.
• Developments in automation may be so fast that by the time the FOT is finalised the automated functions may already be outdated and replaced by better and more advanced ones.
• FESTA takes a scientific approach, but a more industrial approach with short development and evaluation cycles may be needed to study CAVs.
• When evaluation is integrated with development, and tests are kept ongoing, fast iteration of testing steps becomes possible.
Organisational/OperationalEvaluation methodology for small-scale pilotsCARTRER+I 7.2 Common Evaluation MethodologyMicro-FESTA – lessons learnt from small pilots (Chapters 1 and 3.2.) • Full operationalisation of the FESTA methodology can be excessive for small-scale pilots (e.g. less than 3 vehicles, shorter than 4 months, etc.) with respect to capacity and budget.
• To overcome this issue, a condensed version of the FESTA methodology, Micro-FESTA, was produced and utilised to provide a common evaluation methodology for projects too small for a full FESTA approach.
Organisational/OperationalGuidelines for industrial automation tests are neededCARTRER+I 7.2 Common Evaluation MethodologyD4.2 Updated FESTA handbook, including a short version for automation FOTs and inventory of novel methods • Further guidelines about industrial automation tests and freight transport would be needed

Organisational/OperationalNetworking forums help to increase harmonisation and efficiencyCARTRER+I 7.2 Common Evaluation MethodologyD4.2 Updated FESTA handbook, including a short version for automation FOTs and inventory of novel methods • The exchange of knowledge, experience and data will remain of the utmost importance to further our understanding of how FOTs can be conducted as effectively and efficiently as possible.

Organisational/OperationalOperational Design Domain aspects to be considered against accident statisticsCARTRER+I 7.2 Common Evaluation Methodology• For scaling-up the results the Operational Design Domain (ODD) of the tested systems are very important.
• We need to identify on which roads and when these systems can be used and, in addition, to identify the trips and conditions when the people who have the system will decide to use it.
• Statistics and databases that have been used previously for scaling up may not all include relevant elements of ODD. Work is needed to link ODD with the traffic and driving conditions in the whole of Europe.
Organisational/OperationalResearch question sharingCARTRER+I 7.2 Common Evaluation Methodology• Sharing the new types of research questions related to automated driving tests came up frequently.
• Experience from FOTs shows that defining and selecting research questions may be one of the hardest parts of setting up a FOT

Organisational/OperationalDevelopment of corresponding skills in city administrations: CoEXISTR+I 6.3 Workforce development and knowledge enhancement D4.6 Automation-ready Action Plan for each road authority•Although numerous development steps need to be taken before the introduction of highly and fully automated driving, the complexity and dynamics of CCAM require the development of corresponding skills in a city administration.
Organisational/OperationalImportance of agile and adaptive decision making and of legal and policy framework to regulate CAV deployment and service provisionCoEXISTR+I 6.1 Societal, citizen and user needs - for needs-based CCAM solution development and deploymentPeriodic Reporting for period 2 - CoEXist ('AV-Ready' transport models and road infrastructure for the coexistence of automated and conventional vehicles)CoEXist highlights the importance of
•transforming planning practices from the ‘predict then act’ paradigm, towards agile and adaptive decision-making,
•aided by capacity development for local authorities, robust scenario simulations, and cross-sectoral cooperation,
•and the development of integrated legal and policy framework to regulate CAV deployment and service provision
Organisational/OperationalImportance of connected vehiclesCoEXISTR+I 4.2 Connectivity and Cooperative SystemsD4.6 Automation-ready Action Plan for each road authority•Through CoEXist’s results and other research, Helmond understands that AVs need to be connected to other vehicles and the traffic management system in order to contribute to a more sustainable mobility system.
•For this reason, the earlier investments of the city in C-ITS, such as the intelligent traffic lights, will not become obsolete, but on the contrary, are a sound base for future introduction of automated vehicles in the city of Helmond. This has proven a strategic no-regret measure.
Organisational/OperationalImportance of overall coordination to achieve automation readiness: CoEXISTR+I 6.2 Socio-economic and environmental impact analysis and target-based assessment of CCAM benefits
R+I 6.3 Workforce development and knowledge enhancement
D4.6 Automation-ready Action Plan for each road authority• Achieving automation-readiness is a complex process and an overall coordination and an implemented knowledge transfer are essential due to the complexity and variety of tasks and skills.
•Technical, planning and regulatory requirements and frameworks are developing at different speeds.
•Automation-readiness has to be understood as a long-term process depending on various external settings, especially legal frameworks and company driven decisions on technical standards.
•A common understanding and reflection of all currently known chances and risks of automated driving in a city is needed as a basis for a knowledge-based proceeding.
•Concerning the complexity and dynamic an overall working structure guarantees the necessary exchange of information, knowhow, or activities.
•Currently, pilot projects and test operations enable the successive development of skills. At the same time, this approach addresses the framework conditions and standards still open.
•Therefore, appropriate resources must be provided proactively, such as specific experts in various administrative units.
Organisational/OperationalPracticability of automated driving depends on local (road) conditionsCoEXISTR+I 4.3 Fleet and traffic management in a CCAM eco-systemD4.6 Automation-ready Action Plan for each road authority, •Regarding the achievable practicability of automated driving, it should be noted that this depends strongly on local (road) conditions.
•Further specific consideration is required for individual urban structures and road spaces.
Organisational/OperationalGeolocation Requirements HIGHTSR+I 2.1 Environment perception technologies for CCAM

R+I 4.1 Physical and Digital Infrastructure (PDI)
HIGHTS D2.3 Use cases and Application Requirements FINAL• As today a location is provided by a GNSS sub system, the specific requirements are tailored to GNSS capabilities. As we are currently looking at obtaining Geolocation information from a sub system deriving this from many sensors and other systems, the availability of GNSS may become of a much less importance. From the authority side, USA authorities provided the most concrete Infrastructure related Geolocation requirements.
• In Europe, we have some high-level requirements at the moment, but although not yet defined the European Commission is expected to come with similar requirements as provided by the USA.
Organisational/OperationalImpact assessmentInteractiveD7.4, Test and evaluation plans• The main issue for the safety impact assessment was the availability of adequate accident data that allow a detailed reconstruction respectively re-simulation of the accident (lateral conflicts).
Organisational/OperationalDealing with variation across driver types L3PilotR+I 1.1 PilotsD3.4 Evaluation Plan
Chapter 6 Lessons Learned pp. 104–106
• It is important to carefully consider how instructions given to the safety driver, who accompanies the test driver, will influence the behaviour and perceptions of the test driver. Although it would be better from an evaluation perspective to test the systems in their intended ODD, the safety of participants is a greater priority.
• For ADFs which may currently only be operated by a trained professional, ordinary drivers accompany the trained driver and provide feedback from an observer’s view. As the experience of being an observer is quite different compared to using operating the system, data collected this way should be analysed separately.
• Different locales may fall under different regulations concerning the driver types allowed to participate in tests. This may cause challenges for the comparability of results across test sites. Such difference must be considered when interpreting test results.
Organisational/OperationalDealing with variation between test environments L3PilotR+I 1.1 PilotsD3.4 Evaluation Plan
Chapter 6 Lessons Learned pp. 104–106
• There are meaningful differences between and within test environments (geography, infrastructure, drive length, route, traffic, seasonal, lighting and weather differences). These differences will likely affect user experience with ADFs. In L3Pilot, a two-fold approach is used to account for variability between sites:
1. Experimental guidelines have been set to harmonise experimental approaches across sites. The guidelines try to control for study design, test participant selection instruction, experimental protocol + instruction and info given to participants.
2. Where possible, information concerning variations between test sites is collected, which the project-wide user and acceptance evaluation may consider.
Organisational/OperationalHarmonized methods across pilot sitesL3PilotR+I 7.2 Common Evaluation MethodologyD3.4 Evaluation plan
Chapter 6 Lessons Learned pp. 106–107
• It is vital that cross-pilot methods are performed in the same way (tools + protocols) when conducting studies across multiple sites. For example, in L3Pilot the user acceptance questionnaires at all sites were carried out with LimeSurvey, with the only requirement for pilot site staff to translate the questions. This way, the data output was consistent between experiments and pilot sites as it was produced by the same tool, and could easily be merged to a broader consortium-wide database.

• Furthermore, common definitions of performance indicators and driving scenarios were facilitated via a common toolchain for technical and traffic evaluations. This facilitated the use of a common data format, which eased data analysis performed by the research partners. With the aid of a shared code repository accessible to all partners, evaluation sites performed tests with the same algorithms and parameters. The common data format allowed performance indicators to be merged across pilot sites.
Organisational/OperationalProject management RobustSENSER+I 5.2 Robustness and resilienceD1.3 Final Report (Chapter 8.2.1)
• Develop and implement a proper method to monitor the technical progress regularly and ask for input from all consortium partners.
• Risk management activities should be technically oriented, with involvement of all partners on a more regular basis.
• Establish an integration plan early in the project, including early prototypes and demonstrators. Provide pre-prototypes in reviews with the funding authorities.
Organisational/OperationalProject planningRobustSENSER+I 5.2 Robustness and resilienceD1.3 Final Report (Chapter 8.1.1) • Provide overview on the various SW modules and their interfaces. Show feasibility by integrating and demonstrating the complete SW architecture in one of the test vehicles.
• Existing and planned TRL levels need to be communicated thoroughly.
• Software interfaces towards actuators need to be considered as part of the architecture.
• Employ metrics to determine whether a test case succeeds or fails.
• Provide adequate quantifiable baseline and clear evaluation criteria.
Organisational/OperationalExtraordinary challenges due to COVID-19:SAFE STRIP R+I 1.1 PilotsD8.3: Project Final Report (Chapter 21 Conclusions) COVID-19 put a series of barriers in the logistics part of the project:
•Transfer of equipment from site to site delayed,
•user trials were banned in all site countries for a critical period,
•the off-line work itself was also made quite challenging as many employees had to quit their normal daily work routines.
Organisational/OperationalLogistics and operational challenges due to use of equipment in different test sites consecutively: SAFE STRIP R+I 1.1 Pilots
R+I 4.1 Physical and Digital Infrastructure (PDI)
D8.3: Project Final Report (Chapter 21 Conclusions) The fact that SAFE STRIP had to transfer from site to site for the long and complex planned technical validation and user trials was a quite cumbersome condition:
•A series of inevitable delays was created, as apart from the equipment itself, a lot of Partners had to synchronise and collaborate.
•Vehicles, strips and prior to their integration sub-parts of them and RSBs had to travel back and forth and be installed and tested quite often in adverse weather conditions.
Organisational/OperationalProject management SECREDAS R+I 5.1 Cybersecure components and systems• Projects with 70+ partners are very complex and difficult to manage. Still, progress has been good and the cross-domain collaborations are a real result, on top of the technical achievements.
• At the same time, due to the size of the consortium it is very difficult to develop and implement a coherent exploitation strategy for the project. The interests are too diverse and the components (as well as supply-chains) of different levels. So, the exploitation ends up being the sum of individual exploitation activities rather than the exploitation of a clearly identifiable and defined project result.
Organisational/OperationalEthics Board and Ethics Manual needed for projects about CCAM demonstrationSHOWR+I 6.2 Socio-economic and environmental impact analysis and target-based assessment of CCAM benefitsD3.4: SHOW updated Ethics manual + Data Protection Policy and Data Privacy Impact Assessment• An Ethics Manual is needed for projects dealing with CCAM demonstration
• The Ethics Manual gives clarifications about the inner workings of the Ethics Board (EB) and the relations between the local ethics representatives, the partner of the project and the EB. The Ethics Manual touches upon issues concerning ethics in relation to children, incidental findings, incentive schemes and gender. Furthermore, the updated Ethics Manual takes the Covid-19 pandemic into account when it comes to health and safety procedures.
The Ethics Board consists of Core Ethics Board (CEB) and the Local Ethics Representatives (LER). LER are (i) responsible for a demonstration site, (ii) have experience in data collection and/or data management with humans involved and (iii) have experience in preparation and submission of ethical proposals and handling of approvals including compliance to GDPR in relation to vehicle testing.
Organisational/OperationalLimitation in the use of tools, services and technologies for CCAMSHOWR+I 6.2 Socio-economic and environmental impact analysis and target-based assessment of CCAM benefits• The pervasiveness of a technology which many people do not understand and have difficulty to incorporate in everyday daily living activities such as transport/commuting.
• The lack of transparency of the work of other parties necessarily involved such as IT systems’ and control centres’ operators, service providers and other involved providers (e.g. vendors) and their effects on the automation/’driver’- ‘user’ relationship (i.e. both commercial and socio-economic related).
• The difficulty of respecting privacy and confidentiality when third parties may have a strong interest in getting access to electronically recorded and stored personal mobility and transport mode use data.
• The difficulty in ensuring the security of shared personal, localisation, service-use data.
Organisational/OperationalCCAM stakeholders’ wants, needs and prioritiesSHOWR+I 6.2 Socio-economic and environmental impact analysis and target-based assessment of CCAM benefitsD1.1: Ecosystem actors needs, wants + priorities + user experience exploration tools• Wise decision making on planning and design is needed, under a holistic approach, affiliating with the principles of equal access, i.e., not excluding VEC, VRU, PRM and people with sensory or cognitive disabilities and non-connected traffic participants, adopting appropriate interfaces and elaborating an iterative holistic impact assessment of the policies, measures and initiatives taken; based on users’ perception on safety, security, accessibility, comfort, interoperability, ease of use, convenience, (re)liability as well as further added value services provided through automation of vehicles, systems and services;
• There is a need for integration of telecommunication, monitoring and controlling of services in real time through Internet of Things (IoT) and Cooperative Intelligent Transportation System (C-ITS) and also the potential for connection to TMCs for remote intervention at critical situations;
• Business models and exploitation plans promoting long-term partnership agreements and engaging public and private domain (e.g., PublicPrivate Partnerships) should be established, with a clear description of their duties, jurisdiction communication standards, as well as costs and benefits, especially focusing on who is responsible in case of accident or malfunction events;
Organisational/OperationalAgile approach towards the implementation of use cases into simulationsTransAIDR+I 4.3 Fleet and traffic management in a CCAM eco-system D8.2 Results of the meta-analysis (Chapter 4 lessons learned) •Adopting a more agile approach towards the implementation coding of the use cases into simulations would lead to faster release cycles on the one hand, and a better approach to mitigating any unexpected results on the other hand.
•Closely related to the previous point is the fact than an a priori deeper understanding of the simulation logic behind the microscopic traffic simulator SUMO is a requirement. This is a prerequisite for dealing with at-first-sight anomalous results.
Organisational/OperationalDetailed investigation vs. prompt selection of final use cases TransAIDR+I 4.1 Physical and Digital Infrastructure (PDI)
R+I 4.2 Connectivity and Cooperative Systems
R+I 4.3 Fleet and traffic management in a CCAM eco-system
•The selection of the final use cases took quite long. For a large part this was due to the limited availability of information regarding the possible issues in transition areas and the behaviour of (C)AVs, specifically regarding ToC and MRM.
•As a result, it took some time to determine which situations are suitable for studying transition areas as well as how the simulations should be parameterised to cope with the multitude of combinations of factors.
•In hindsight, we perhaps spent too much time on trying to find information regarding our ‘problem’ (i.e. mixed traffic in transition areas) and should have decided more quickly, it is what it is.
•A quicker selection of the final use cases would have been more fruitful, as a lot of details and issues regarding the different cases would arise once implementation began, though the careful considerations beforehand were of value as well.
Organisational/OperationalImportance of cooperation between projects in the same area TransAIDR+I 4.1 Physical and Digital Infrastructure (PDI)
R+I 4.2 Connectivity and Cooperative Systems
R+I 4.3 Fleet and traffic management in a CCAM eco-system
•A more open and sharing approach should be followed on the EU level, but it is up to the EC to instigate programmes and opportunities (or even obligations) to exchange and align results between different projects.
•This would allow to compare the results and identify commonalities between the traffic management scenarios and (C)AV vehicle behaviour.
Organisational/OperationalUpDriveD10.3 Periodic technical report• Integration is hard and time-consuming. Easily underestimated
Organisational/OperationalUpDriveD10.3 Periodic technical report• Many shorter development and integration/evaluation cycles lead to faster learning and better results than big iterations
Organisational/Operational
Technical
Integration in existing traffic management centres
TransAIDR+I 4.1 Physical and Digital Infrastructure (PDI)
R+I 4.3 Fleet and traffic management in a CCAM eco-system
D8.2 Results of the meta-analysis (Chapter 4 lessons learned) •The integration in existing traffic management centres would be beneficial to better address future topics like individual advice generation, but also to include more macroscopic strategies as possible countermeasures.
•Here, also scalability plays an important role, from larger communication ranges using range extension technologies (hopping, linking of RSUs) to advice generation on a street, local, regional or even a national level.
Organisational/Operational
Social
Importance of impact assessment methodologyICT4CARTR+I 6.2 Socio-economic and environmental impact analysis and target-based assessment of CCAM benefitsD8.4 : Impact assessment (not ready yet) • Lack of a proper Impact assessment methodology when it comes to compare advance CCAM services. For example, crossing an intersection when external sensors contribute or not to the perception build of the car is way too complicated to be evaluated from the impact assessment and passengers perceived safety point of view. This is even more complicated when a small amount of trials can be performed
SocialSharing user questionnairesCARTRER+I 7.2 Common Evaluation MethodologyD4.2 Updated FESTA handbook, including a short version for automation FOTs and inventory of novel methods • A common approach to measuring user acceptance could help to compare studies and to create a better understanding of acceptance of CAVs based on several studies.
• It would also be helpful if questionnaires and interview questions were shared amongst the FOT community.
SocialUser needs and acceptanceFABULOSR+I 6.1 Societal, citizen and user needsD5.6 Future application and impact of project results and learnings• Overall user acceptance is positive, but real needs evaluation require further investigation, and testing, notably in circumstances without any on-board steward.
SocialConsideration of parallel changes in the mobility systemL3PilotR+I 6.2 Socio-economic and environmental impact analysis and target-based assessment of CCAM benefitsD3.4 Evaluation plan
Chapter 6 Lessons learned p. 107
• If the evaluation considers the future, expected influence from other mobility developments should be included. These factors may limit the extrapolation of ADF impacts far into the future. L3Pilot solution was to apply snapshot approach for the socio-economic impact analysis.
SocialExtrapolation of user experience and acceptance to real world conditions
L3PilotR+I 1.1 PilotsD3.4 Evaluation plan
Chapter 6 Lessons Learned pp. 104–106
The main aspects to consider when generalising findings to real world conditions include: maturity of the system, test environment and driver type. More specifically, these are:
• Prototypical nature of human-machine interactions (HMI) and automated driving (AD) control systems, which may still be under development, and therefore result in occasional unpleasant driving experiences and interactions. These should be considered as they affect user experience and ultimately acceptance. The practical result of this is that L3Pilot featured few direct behavioural evaluations of ADFs or their HMI. Instead, focus was placed on the indirect impacts of ADF use.
• Pilot participant drivers were not be able to use the piloted systems in their daily travel. The requirements of the testing environment will likely lead to behaviours that are different to a real-world case. The existence of such differences ought to be considered in such tests.
• In tests where a trained safety driver must accompany the main driver, recorded perceptions and behaviour are likely to be influenced by their presence and/or their task to ensure safety of the test. Instructions given to the safety driver must be carefully considered so that the potential confounding effect can be taken in to account.
SocialAppropriately framing the boundaries of influence of CATS use cases, applications and interventionsLEVITATER+I 7.2 Common Evaluation MethodologyD4.3 Feasible paths of interventions
Chapter 3.1 p. 20
• Through dialogue with city representatives, it was found that CATS use cases, applications and interventions can relate to both factors influencing visions as well as policy interventions designed to achieve visions. A strict distinction between the two is thus not always possible to make, as was initially attempted.
SocialAssessing the completeness of KPIs to measure vision achievement

LEVITATER+I 7.2 Common Evaluation MethodologyD4.1 Definition of quantified policy goals
Chapter 3.4 (p. 26)
• As the research process progresses, it may be necessary to ensure that the selected KPIs used to measure the achievement of a vision are complete and independent.
• In LEVITATE, potential sets of KPIs were evaluated based on completeness (E.g., where there any important aspects not covered?) and independence (e.g. were the selected KPIs sufficiently different from each other?).
• Shortcomings concerning completeness suggests an incomplete list, while shortcoming concerning independence suggests that there may be redundant KPIs in the list.
SocialComparing indicators between cities and regionsLEVITATER+I 7.2 Common Evaluation MethodologyD4.1. Definition of quantified policy goals
Chapter 3.3. (p. 25)
Indicators between cities or regions can only be compared if they:
• Either express quantities that have the same scale across regions of different size (e.g. distance to nearest accessible public transport stop).
• Are formulated as a suitable ratio (e.g. number of casualties per year and per 100,000 residents).
Furthermore:
• When evaluating existing data from different regions (or time periods), it might be challenging to map these to the precise definition of the desired indicator.
• Targets and KPIs sometimes are expressed in trends over time (for example, ‘a 15% reduction in pedestrian fatalities over the next five years’) or in comparisons with other jurisdictions (for example, ‘reduce crashes on country roads to below the national average’) – which adds another level of complexity for comparability.
SocialDealing with correlation between factors influencing visions and CATS/AvsLEVITATER+I 7.2 Common Evaluation MethodologyD 4.3 Feasible paths of inteventions
Chapter 3.3. p. 24
• Relationship between factors influencing visions and CATS / AVs not always clear, and there may be correlations between factors. For example, electrification of vehicles may impact environmental goals more than automation, but most AVs are likely to be electric. Special care must be taken when interpreting such correlated factors.
SocialEnsuring a comprehensive set of indicators covering all relevant dimensionsLEVITATER+I 7.2 Common Evaluation MethodologyD 4.3 Feasible paths of interventions
Chapter 3.2. p. 20
• Simple visions (defined as a small set of key targets and indicators) are not suitable for direct (“brute-force”) optimisation approaches, as the simplified nature of such models may lead to extreme means to achieve vision goals (e.g. ban all traffic). To avoid such situations, a more comprehensive set of indicators must be employed (e.g. in LEVITATE, indicators within four dimensions: safety, society, environment and economy were considered).
• Optimisation approaches will then consider the broader set of indicators implicitly while trying to improve a smaller set of CATS-related key indicators.
• Consideration of all dimensions while selecting and prioritising influential factors and policy interventions is also important, to avoid the selection of interventions which might result in benefits for one dimension, but disadvantages for the others.
SocialImportant factors to consider when estimating generalised costs of travel associated with vehicle automationLEVITATER+I 7.2 Common Evaluation MethodologyD4.4 Detailed list of forecast scenarios applicable forecasting methodologies and output variable
Chapter 3.2 (p. 22)
• The capital cost of vehicles, including the cost of vehicle purchasing the and maintenance/possession. Such costs can be converted to cost per kilometre by assuming a depreciation rate, i.e. a rate for the fall in the value of a vehicle as it gets older. Automated vehicles are, in general, assumed more expensive to buy than current vehicles.
• The operating cost of vehicles. These include expenses incurred when driving a vehicle. Operating costs are generally expected to become lower with vehicle automation, as vehicles will be operated more efficiently
• Changes in travel time. Unless there is a perfect rebound effect, the increase in road capacity brought about by vehicle automation is anticipated to reduce travel times.
• Changes in the valuation of travel time. The studies made so far, reviewed in deliverable D3.2, suggest that the value of travel time savings will be reduced.
SocialIssues to consider when estimating primary and net impacts on accidentsLEVITATER+I 7.2 Common Evaluation MethodologyD3.2 Methods for forecasting the impacts of CATS (Chapter 7.3.5, p. 104)
• There are different dose-response curves for different types of accidents and different groups of road users.
• There is a category of accidents that will not be affected by vehicle automation, i.e. accidents involving vehicles and road users that will not be automated.
• To estimate net impact, one should adjust for the expected increase in traffic volume.
SocialLimitations for defining quantitative pathways to achieve visionsLEVITATER+I 7.2 Common Evaluation MethodologyD4.3 Feasible paths of interventions
Chapter 7.2. P. 63
• Uncertainties in the expected impacts of AV uptake is a challenge for defining quantitative pathways to achieve visions. Several different scenarios could be further considered here for further investigations and each of them would result in different policy interventions needed at a particular time.
• Qualitative impacts are also hard to predict because optimal use of AVs might require a different way of planning for which existing tools and data are inappropriate.
SocialMethod for visualising and comprehending the distance between the current situation and the target vision via dimension reductionLEVITATER+I 7.2 Common Evaluation MethodologyD4.3 Feasible paths of interventions
Chapter 3.2. P 20–21
• Relationships between indicators (how might indicator X change if value of indicator Y is changed) may be estimated by mapping geographic entities (e.g. cities) to specific points in time within an abstract indicator space with reduced number of dimensions. For instance, a PCA can be used to reduce indicator dimensions and track change over time.
• Vision points based on targets can also be added to the space to illustrate the gap between today, business as usual trajectories and the vision. The distance between these points can be used to derive a “Multi-dimensional difference vector” to indicate what must be done to achieve the target and identify transformation corridors. Potential pitfalls include lack of some significant data, which may skew conclusions.
SocialMethod used by LEVITATE for selecting relevant policy goalsLEVITATER+I 7.2 Common Evaluation MethodologyD4.1 Definition of quantified policy goals
Chapter 3.5 (p. 26)
• Organisation of goals into a small set of dimensions (highest-level goals)
• Breaking down high-level goals (long-term visions) into objectives, targets and finally assign them KPIs
• Consider correlation of CATS impacts to KPIs – how far can a KPI be influenced by CATS?
• Consider the practicality of indicators – how easily can they be measured and compared?
• Consider the completeness and independency of goals and indicators
SocialPreviously used methods for estimating and predicting impacts of connected and automated drivingLEVITATER+I 7.2 Common Evaluation MethodologyD3.2 Methods for forecasting the impacts of CATS (Chapter 9.1) • Based on a literature review, both retrospective and future-oriented methods were deemed relevant for predicting impacts of CATS based on previous applications. Traffic simulation is the most frequently used method and is considered to be among the most suitable for estimating future impacts of CATS. Cost analyses and willingness-to-pay studies were considered relevant for providing bases for cost-benefit analyses of CATS, but only few such studies are available
SocialRecommended procedure for predicting impacts of CATS LEVITATER+I 7.2 Common Evaluation MethodologyD3.2 Methods for forecasting the impacts of CATS (Chapter 7) • Define use case or traffic environment type for which impacts will be predicted. Distinction should be made between urban, rural and motorway-traffic environments.
• Decide which impacts to include in analysis. Policy objectives set by stakeholders should be used as frame of reference. Should aim to identify impacts of potential unintended side-effects of policy objectives, e.g. rebound effects
• Estimate primary impacts (e.g. Functional relationships (dose-response)). Impacts should be estimated with and without inclusion of behavioural adaptation effects. This will help understand how behavioural effects influence total benefits of CATS
• Estimate rebound effects (e.g. induced traffic from reduced cost of travel, which may counteract intended benefits of CATS)
• Estimate net impacts after correcting primary impacts for rebounds
• Assess uncertainty and develop best and worst-case scenarios. Scenarios can be based on upper and lower limits from functional relationships. The worst case is that all impacts take on their smallest values, the best case is that they take on the largest values
SocialSuitability of dose-response curves to measure impacts of CATS in existing literature LEVITATER+I 7.2 Common Evaluation MethodologyD3.2 Methods for forecasting the impacts of CATS (Chapters 8.1 + 9.2) ´• The transition to a transport system primarily based on CATS will likely take an extended period of time, and the impacts are likely to develop gradually in line with increasing market penetration rate of automated vehicles. Therefore, an approach considered particularly suitable for assessing impacts is the fitting of dose-response curves. These curves estimate impacts with continuous functions, with the level of automation or market penetration rate as independent variables and desired impact as the dependent variable.
• Impacts of CATS have usually been modelled with dose-response curves in past literature, with market penetration rate of automated vehicles as the dose and impact size as the response.
-Some impacts will take the form of a continuous function, some may resemble stepwise changes.
SocialA set of plausible future scenarios for cities in EuropeMOMENTUMR+I 6.1 Societal, citizen and user needs
D2.1 New Mobility Options and Urban Mobility High -level development scenarios for cities in Europe (SSP), developed by the climate change research community, can be used as a mainframe to develop mobility scenarios by adding mobility parameters. They result in 4 high level scenarios :
• Scenario 1 : Mixed compact cities in a sustainable Europe
• Scenario 2 : stangnant individualist cities in a nationalist Europe
• Scenarios 3 : segregated green cities in an unequal Europe
• Scenarios 4 : sprawling technological cities in a vibrant Europe
SocialLearning about behavioural models to increase driver awarenessSIMUSAFER+I 6.1 Societal, citizen and user needs - for needs-based CCAM solution development and deploymentSIMUSAFE infographic-A thorough knowledge of different and reliable behavioural models of different road users on critical traffic circumstances will allow the design of effective driver training modules. Overall SIMUSAFE has created initial steps towards the creation of possible new training modules on utilisation of biometrics.
TechnicalTrial framework correlation in EU and China5G DRIVER+I 1.2 Field Operational Trials (FOTs)D4.4 Final report of V2X trials_v1.0 (Section 3)• Road ITS spectrum and V2X technology paths in the EU and China are different (i.e. Coexistence of two V2X technologies-ITS-G5 and LTE-V2X PC5 only exists in Europe), but can be harmonised under a joint EU-China V2X trial framework that guides joint trial activities with joint trial methodology and plan.
TechnicalNetwork synchronization 5G MOBIXR+I 4.2 Connectivity and Cooperative SystemsD3.3: Report on the 5G technologies integration and roll-out • Typical connectivity-oriented projects aim at single network/operator setups where device synchronization is solved through SOTA solutions e.g., GNSS and NTP/PTP.
• However, in cross-operator setups, synchronization is a new issue, emerging when service continuity is important.
TechnicalLegal issues
Human vehicle integrationAutomation in close distances
Automation in urban scenarios
Automation in highway scenarios
Evaluation
AdaptIVe (FP7 project)D1.0 Final project results (chapter 12.1)• Lessons learned in bullet points in the deliverable
TechnicalFlexibility of evaluation framework to adapt to differences in technology maturity of demonstratorsADAS&MER+I 3.1 Future-proof methodologies and tools for validationFinal report ADAS&ME (D10.4) • Issues became apparent with differences in maturity of technology available during the testing stage concerning the original plans.
Due to this ad-hoc solutions often needed to be developed within a short timeframe before the start of the evaluations (with the use of valuable time and resources detracting from the effective implementation of the evaluations). ).
• Improvement in the flexibility of the evaluation framework could have provided some other options.
TechnicalKey issues with failure of demonstrator vehicles and integrated technologyADAS&ME• When a demonstrator representative was not present issues often became difficult and time-consuming to resolve, and for integrated technology, issues were difficult without input from relevant partners. This caused numerous instances of delays and/or missed data collection. In response to this, it should have been mandated that a demonstrator representative was available at the evaluation site for the duration of the testing. Also provision of user manuals.
TechnicalTake into account language specifications when designing the systems, studies, validation framework, test proceduresADAS&MER+I 3.4 Specific HMI testing and validation procedures, methods and tools for higher levels of automation• During the development of both the sensing technology and the vehicle demonstrators, systems were developed with the partner language. This led to issues with the need to recruit participants in the test location with a native level in a foreign language.
• This was a limitation for the number test participants and on other recruitment criteria (i.e. representative gender balance, driving experience). Consideration should be given to the specification of the testing environment before the development to ensure that content could be tested with an optimal test procedure.
TechnicalOperational boundaries for AIAI4DIR+I 5.4 AI for situational awarenessD1.13 Definition of requirements for AI in transportation • Conventional safety logic and algorithms are very useful in double-checking results coming from neural networks. In other words, neural network components should often be surrounded with conventional safety logic, providing operational boundaries.
TechnicalImpact of COVID19AutoDriveR+I 6.1 Societal, citizen and user needs - for needs-based CCAM solution development and deployment• COVID-19 pandemic nevertheless causes lot of trouble and additional management effort.
TechnicalBuilding High-accuracy Positioning SolutionsAutoNet2030 (FP7 project)D1.3 Public Final Report (Chapter 7.1) • Τhe implemented positioning technology is considered to be very well matching the positioning accuracy requirements of automated driving. Future efforts may include addressing issues related to overhead obstacles or urban canyons.
TechnicalBuilding HMI Applications for Connected VehiclesAutoNet2030 (FP7 project)D1.3 Public Final Report (Chapter 7.1) • Challenges with the use of Head-up display. The results gave positive feedback on the applicability of an Android device for HMI purposes.
TechnicalDistributed/semi-decentralized Control of Automated VehiclesAutoNet2030 (FP7 project)D1.3 Public Final Report (Chapter 7.1) • An iterative development framework is needed for distributed control. Cooperative control is highly dependent on perception and communication components.
TechnicalIntegrated Systems for Future Vehicle AutomationAutoNet2030 (FP7 project)D1.3 Public Final Report (Chapter 7.1) • When working with standardized interfaces and isolated modules, one must focus on architectural questions and specifications of the interfaces very early in the project.
TechnicalPerception Layer to Cope with Different Vehicle PlatformsAutoNet2030 (FP7 project)D1.3 Public Final Report (Chapter 7.1) • Additional plausibility and consistency checks should be implemented in parallel to the perception in order to handle complexity and safe detection of potential failure states of the whole system.
TechnicalV2X Communications for Cooperative AutomationAutoNet2030 (FP7 project)D1.3 Public Final Report (Chapter 7.1) • Τhe use of tools for monitoring of experiments by means of V2X communication is highly beneficial. Such tools would need to be developed at an early stage of the project.
TechnicalIoT in vehicle platformAUTOPILOTR+I 2.1 Environment perception technologies for CCAMD1.6 Final open IoT vehicle patform specification • The core functionalities of such a platform should include and/or consider: interoperability, service-based, context-awareness, data-management, remote management, security and privacy, use open standards, define APIs, manage events through analytics and UI
TechnicalIoT security in ADAUTOPILOTR+I 5.1 Cybersecure components and systems
R+I 5.2 Robustness and resilience
D1.10 Final-specification of Security Privacy for IoT enhanced AD
• It is important to include a security-by-design method. This way it will be easier to incorporate security at the design phase, provide advanced security updates and vulnerability management, build on proven security practices, prioritize security measures according to potential impact, promote transparency across IoT and connect carefully and deliberately.
• Moreover, any new project shall include a high level risk analysis during the initial project proposition to better deal with security.
TechnicalRequired standards for InteroperabilityC MOBILER+I 4.2 Connectivity and Cooperative SystemsD6.2 Technical Validation Report• C-ITS large-scale deployments are a technically viable reality and are already happening, but standards to ensure interoperability and data access must be put urgently in place, so the market uptake is not hindered and full deployments are too delayed.
TechnicalAutomated processing of test dataCARTRER+I 7.2 Common Evaluation MethodologyD4.2 Updated FESTA handbook, including a short version for automation FOTs and inventory of novel methods • A main challenge in data acquisition, data processing and data analysis is establishing automated processes.
• Because of the large amount of data manual analysis may not be possible. Automated processes have to be implemented, which will provide analysis results without constant interaction with data analysts.
• Ideally the data is automatically classified into single driving scenarios/situations, which can be further processed in terms of parameter analysis, correlation, etc.
TechnicalExtensive long term testing in live traffic C-ITS corridor• The gap between theory and the practical reality can only be narrowed by testing, testing and testing in live traffic between international partners.
• Intensive short term testing is great, extensive long term testing is indispensable in finding the real issues.
TechnicalGLOSA speed advice C-ITS corridor• A maximum speed must be included in the advice speed calculation so that the in-car speed advice does not exceed the maximum speed. If this is missing, the speed advice should not be given. Time synchronization is also crucial for the service.
TechnicalHybrid communication C-ITS corridor• ITS G5 and Cellular (short- and longrange communication) work well together. Thanks to the use of exactly the same data and the exact same format over ITS G5 and cellular, no serious integration problems arise in-car. The advantages of the one compared to the other are still not clear. More research will have to be conducted in a much more focused way using solid research questions and using international stable
TechnicalInfrastructure design guidelines C-ITS corridor• The costs of one-off / stand-alone C-ITS infrastructural deployment are very high. Combining C-ITS deployment with existing works can drastically reduce the costs.
TechnicalLane sequence C-ITS corridor• In some countries, the lane sequence may be mirrored compared to what is common in continental Europe. As a result, the lane sequence does not correspond to reality and warnings are sometimes displayed in the wrong lane.
TechnicalThe Dutch MTM-ESB chain C-ITS corridor•The ESB client specification should contain a warning regarding the method used in combination with a firewal / proxy, with a reference to RFC6202.
TechnicalOpen standards for the video data sets: Cloud LSVAR+I 5.5: System architecture for data sharingD6.6 Initial Standardisation planOpen standards for the video data sets:iIn this project a draft of open video dataset standards was developed. The performed gap analysis activities resulted in defining the necessary requirements for a suitable/following video data format/standard:
• multi-dimensional Objects with time dimension
• describe Events/Actions
• Connection with Ontologies to enable reasoning
• Good storage/streaming trade-off
• Efficient parsing into XML, JSON, etc.
• Lightweight API to translate app output ready for evaluation
TechnicalImportance of physical infrastructure support CoEXISTR+I 4.1 Physical and Digital Infrastructure (PDI)D4.6 Automation-ready Action Plan for each road authority•Lessons learnt from CAV testing and the CoExist case studies suggest that some technical solutions clearly would benefit from physical infrastructure support.
•However, improving the physical infrastructure in a large part of the road network is much longer process than expanding digital services.
TechnicalNecessary improvements before productionDENSER+I 2.1 Environment perception technologies for CCAMFirst full fusion sensor solution enables all-weather assisted driving• More bad weather simulation data
• Better automatic weather recognition
• Increased sensor hardware performance
• Hardware development
TechnicalThe added value of robotic technology and cognition for Automated DrivingDreams4CarsR+I 5.4 AI for situational awarenessD7.3 Final Report• Dreams4Cars has compared the driving agents evolved by the simulation technology to a baseline agent from the AdaptIVe EU project in driving automation, verifying the added value of the robotic technology (with target Technology Readiness Level 6) and the opportunities offered by cognition in automated driving.
TechnicalCross-domain methodology for Verification + Validation Enable S3R+I 3.1 Future-proof methodologies and tools for validationD3.2.2 VV Methodology• Complementary to research projects focusing solely on the automotive domain, such as PEGASUS or RobustSense, ENABLE-S3 has worked on developing a set of technology bricks (toolbox), creating a tailored validation toolchain based on new and existing technologies across several domains.
• Depending on the function under test and the purpose of the test, the requirements for the tool chain varies. ENABLE-S3 results support the required flexibility by providing a set of technology bricks and a common reference architecture for ADAS/AD system validation along the value chain.
TechnicalModular structureEnable S3R+I 3.1 Future-proof methodologies and tools for validationD3.4.1 Platform specification • A modular structure allows the reuse of different technology bricks by various OEMs, Tiers, component suppliers, academia and research institutes, tool suppliers and of course also for different applications. This modular und reusable structure enables faster and more efficient tool chain tailoring independent of the tool supplier, avoiding vendor lock-in and substantial costs for switching between products and services.
• The Generic Test Architecture is further aligned with other domains such as maritime, rail, aerospace, health, enabling experience and know-how exchange, especially for application independent technologies, such as co-simulation platforms and the integration of simulation and real-time components (hardware).
TechnicalStandardization is requiredEnable S3R+I 3.1 Future-proof methodologies and tools for validationENABLE-S3_D5.3.2_Report on standardization work• The modular structure can only be used in an efficient way if it supports standardized interfaces.
• To ensure the sustainability of the achieved results, ENABLE-S3 contributed significantly to the standardization of OSI and two other specifications (OpenDRIVE, OpenSCENARIO).
• As a result, standardization of developed and improved industry-defined interfaces allows to integrate existing tools at customer sites and to save significant money in the reuse of well-established tools. Moreover, the handover of results and tight cooperation with standardization ensures that the achievements are distributed in the automotive community, maintained and extended to future requirements.
TechnicalUse CasesENSEMBLER+I 3.2 EU wide database of relevant scenarios for validation• Technical alignment on use cases and specifications is a time-consuming process
TechnicalEvaluation concept HIGHTSR+I 7.2 Common Evaluation MethodologyHIGHTS D6.3 – Final platform description A novel evaluation concept has been developed in HIGHTS, based on an extended graph SLAM algorithm, which allows a joint processing of the recordings of multiple vehicles. From this approach, first results have been generated for recordings of the final Helmond test campaign and promising results have been achieved.
TechnicalSuccessful validation of three cooperative and/or hybrid localization algorithms HIGHTSR+I 3.1 Future-proof methodologies and tools for validation

R+I 3.4 Specific HMI testing and validation procedures, methods and tools for higher levels of automation
HIGHTS D6.3 – Final platform description • First of all, we have shown (online) in a simple roundabout context that CMDS could guarantee not only a level of accuracy comparable to that of standard GPS (i.e., on the order of few meters), but also –and maybe more noticeably- a more stable quality of service, as long as the vehicle keeps running under the coverage of a few ranging-enabled elements of infrastructure (typically, Zigbee- or IR-UWB-enabled georeferenced anchors). CMDS could thus represent a low-complexity and ultra-fast alternative to conventional GPS in practical satellites-denied environments.
• Then, on a portion of highway, we have shown (offline) that VA-CLoc could benefit from V2V ITS-G5 CAM messages (sent by reliable neighbouring vehicles, seen as Virtual Anchors) and accurate V2V IR-UWB range measurements to achieve a quasi-constant level of accuracy around 0.2 m - 0.3 m (in compliance with the initial target claimed at the beginning of the project), without being sensitive to input GPS errors larger than 2 m. This result thus confirms the capability of V2V-aided cooperative localization to improve both localization accuracy and resilience.
• Finally, still on the same portion of highway, we have demonstrated (offline) that ICP could reduce the 2D location error by 65 % in comparison with regular GPS, especially when the latter experiences dramatic outliers (i.e., very large errors beyond 10m). This illustrates the tangible benefits brought by both cooperative fusion approaches and precise features/objects detection around the vehicle (Lidar-based detection in our case), enabling again better localization resilience.
TechnicalMisunderstanding between ICT and ITS partners of the MEC use and definition.
ICT4CARTR+I 4.1 Physical and Digital Infrastructure (PDI)D3.3: IT Environment Specifications and Architecture


• ICT partners (vendors and MNOs) have a 3GPP like interpretation of how MEC can be deployed and operated in an CCAM environment, where one or two instances are enough to provide the necessary QoS of a great area. On the other hand, ITS partners (OEMs and road operators) assume the inclusion of MEC entities in a larger scale per intersection or so.
• A more standardized and well defined use of MEC to provide low latency and high reliability should be the focus of future R+I iniatives.
TechnicalMultiple Object TrackinginDRIVER+I 2.1 Environment perception technologies for CCAMD3.4 Performance Evaluation • MOT heavily affected by errors in data to be fused and a one-to-one correspondence can be found between the errors by the MOT and the worsened quality of its input. When the quality of inputs is good, the MOT behaves as expected
• An input data induced unexpected behaviour of the MOT will however have a conservative effect, whereby no object detected by the sensors is ignored.
TechnicalPositioning: Unsolved Problem inDRIVER+I 2.1 Environment perception technologies for CCAMD3.4 Performance Evaluation • Lane assignment - conflict between trusting the highly influential lane detections to achieve the desired lane-positioning-accuracy and falling for the sensors outlier measurements
TechnicalRole of EGNSS inDRIVER+I 2.1 Environment perception technologies for CCAM•Recognition of the unique role of absolute E-GNSS positioning in data fusion for autonomous driving
• The reasoning of the possible specific role of the European GNSS over the other satellite systems
TechnicalDependence of effectiveness of measure on the specific scenario and the way of implementation INFRAMIXR+I 4.3 Fleet and traffic management in a CCAM eco-systemD.5.3 Evaluation, impact analysis and new safety performance criteria, Chapter Conclusions, Based on the obtained results, conducted evaluations and the gained experience, there are clear observations made for the set of methodologies and software tools as well as their capabilities and limitations for analyzing the specified mixed traffic scenarios involving automated and manual driven vehicles:
•These analyses indicated some obvious implications of the analyzed measures using the conjectured C-ITS messages and the INFRAMIX Management Center.
•However, not all the measures and KPIs were effective in analyzing their respective impact on the traffic flow.
•Also, not every measure was found to be as influential in the same way for every scenario.
•The overall observation is that the effectiveness of the measure over the baseline traffic flow is quite dependent on the specific scenario, as well as the way and the means that the measure is implemented.
TechnicalDynamic lane assignment does not seem an appropriate measure for mixed traffic control. INFRAMIXR+I 4.3 Fleet and traffic management in a CCAM eco-systemD.5.3 Evaluation, impact analysis and new safety performance criteria•Concerning the impact of Dynamic Lane Assignment on traffic efficiency, the effect is positive for AVs, especially when the demand for traffic is scaled down to 60% and the penetration of AVs is 25%, but not for the rest of the traffic.
•If areas around on / off ramps are included in the assignment logic, the results are a little bit higher than the other case.
•The analyzed KPIs showed neither an improvement in throughput nor in TTC (time to collision –used for safety analysis).
•KPIs such as the delay time or total travel time decreased significantly, while TTC deterioration was about twice as high as for the bottleneck simulations.
•Therefore the DLA does not seam an appropriate measure for mixed traffic control
TechnicalImportance of standardisation INFRAMIXR+I 4.2 Connectivity and Cooperative SystemsD4.2 Demonstration phase and data delivery

An impressive example of standardisation was shown in the ITS-G5 communication within the Spanish tests:
•ITS-G5 equipment in the AAE test vehicle equipped with ATE technology could communicate successfully with the INFRAMIX ITS-G5 Road Side Units during the complete Spanish test runs, and could furthermore communicate standardised day1-services with other AAE ITS-G5 road side equipment (installed outside of the INFRAMIX project) without special configuration.
TechnicalImportance of sufficient pre-testing including the whole communication chain INFRAMIXR+I 4.2 Connectivity and Cooperative SystemsD4.2 Demonstration phase and data delivery

•An important lesson learned during field trials was that sufficient pre-testing is a key factor for successful field tests and demonstrations.
•It is important to highlight that the whole communication chain should be included in such pre-tests in the field.
•The Spanish demonstration, being the first one that took place, acted as a very complete pre-test for the preceding demonstration at the Austrian test site.
TechnicalRaw data managementinLane• The management of the recorded data during the pilots requires extra efforts that are usually not foreseen in the project preparation phase.
TechnicalinLane• Digital twin for running simulation variances of real recordings is key for ensuring the proper integration. However, when fusing GNSS with computer vision is difficult to cope simulators capable of generating both correlated elements.
TechnicalImportance of policy parameters and traffic management objectives for selecting appropriate toolsMAVENR+I 4.3 Fleet and traffic management in a CCAM eco-systemDeliverable D7.2 Impact Assessment – Technical Report (Chapter 6 conclusions) The lane advice use case, however, did not perform as expected for the single intersection:
•Normally, the controller has a bias for preventing stops due to platoon dispersion.
•However, when platooning and lane advice are combined, this effect does not occur and the algorithm automatically follows its setpoint to give high priority to reducing the delay.
•Due to the balance shift from stops to delay, the CO2 emissions increased. •Therefore, traffic managers should take into account that recalibration of the policy parameters is probably required when using all MAVEN use cases.
•At the same time, the different algorithms can aim at contradictory objective functions, for example, the number of stops versus delay.
•The traffic managers should choose the traffic management objectives first and based on that select the most suitable tools (use cases).
TechnicalFeatures and performance vs. computer power availableMuCCA R+I 2.2 Safe and reliable on-board decision-making technologiesFinal report• The hardware had an adequate performance for our final demonstration, but probably some struggles would have been found if the full perception system originally expected to implement, and human driver model, were running simultaneously. A performance upgrade is already in progress however.
TechnicalTechnology readiness and availability.MuCCA R+I 2.1 Environment perception technologies for CCAMFinal report• The project has struggled to find a suitable of-the-self solution for a perception system that met the project requirements in terms of performance (long-range motorway speed), cost and maturity. The sensor market has been evolving quickly but there´s no cost-effective solution yet compared to other sources of data to feed localization (i.e.: enhanced GPS and comms).
TechnicalPRoPART positioning solutionPRoPARTR+I 4.2 Connectivity and Cooperative SystemsPRoPART Positioning Manager
PRoPART Final Demonstration
TechnicalRole of Galileo in PRoPART PRoPARTR+I 4.2 Connectivity and Cooperative SystemsPRoPart Role of Galileo Overview
PRoPART Final Demonstration
TechnicalStandardisation of messages for connected vehiclePRoPARTR+I 4.2 Connectivity and Cooperative SystemsCooperative Perception Concept
PRoPART Final Demonstration
• Highly beneficial to standardize the Cooperative Perception Message for connected automated vehicles to exchange sensor data with each other and with infrastructure, e.g. road monitoring sensors.
TechnicalUWB ranging as a complement to GNSS PRoPARTR+I 4.2 Connectivity and Cooperative Systemshttp://propart-project.eu/wp-content/uploads/2019/12/6_PRoPART_Final_Demo_UWB_Ranging_Concept_CEIT.pdf
TechnicalAntenna and cabling topology ROADARTR+I 4.1 Physical and Digital Infrastructure (PDI)
R+I 4.2 Connectivity and Cooperative Systems
• The roof of the truck is not considered a feasible spot to plug the antenna, but the basic mirror should be preferred.
• Few recommendations are considered as best practice: keep connection with coax cable as short as possible, for all components not located in driver’s cab protect from splash water, the length of digital connection of RF-modules with communication unit should be extendable up to 10 m (for use in city buses and coaches), antenna subsystem and architecture should work according to the current standardization of V2X via WLAN 802.11p
TechnicalIntegration of communication techniques in trucks

ROADARTR+I 4.2 Connectivity and Cooperative SystemsROADART - D7.3 Final Demonstration • Diversity communication techniques can be easily integrated in trucks and applied to T2T and T2I communication links. Moreover, the great benefits depicted in the achieved KPIs come with low complexity, straightforward installation and accommodation in the side mirrors, and without any modifications to the existing standards.
TechnicalImplementation challenges in a new fieldSAFE STRIP R+I 4.1 Physical and Digital Infrastructure (PDI)D8.3: Project Final Report (Chapter 21 Conclusions) The system itself has been from the vision phase a very challenging one.
•Although the technical management designed, applied and monitored from the very beginning a very tight plan, the implementation challenges have been inevitable, as this was a new field for all members involved despite the high expertise of the Consortium in their fields.
•System architecture and specifications had to change over the course of the project, the integration and iterative testing planning as well.
•A lot of challenges were met in the actual development and mainly integration of the system elements.
TechnicalEvaluation of cybersecuritySAFERtecR+I 7.2 Common Evaluation Methodology• Cyber-security evaluation is costly both in terms of monetary resources and time, especially for the automotive case that calls for high assurance levels. The automotive industry must define agile and cost-efficient processes targeted for the automotive ecosystem. Importantly, to devise assurance arguments of high credibility, the scope of any relevant approach needs to be extended at the system-level (of the connected vehicle and infrastructure). That is a significant challenge (along numerous fronts) for future research.
TechnicalInclusion of Protection Profile documentation in the connected vehicle systemSAFERtecR+I 5.1 Cybersecure components and systems• Protection Profiles (PPs) are documents (of a certain structure and terminology) specified in the context of the Common Criteria (CC) standard to formally define (among others) the involved security functional requirements, in an implementation-independent way. PP refer to certain modules under consideration (called Target of Evaluation) and enjoy recognition of a broad audience (even if not considered in the full CC context). It is thus important to develop such documents for components of the connected vehicles system, not currently covered.
TechnicalSecurity assurance methodologySAFERtecR+I 7.3 Test Data Exchange Framework• Newly introduced Security Assurance approaches require carefully selected concepts (to align with the automotive needs and if needed, follow well established/standardized frameworks) and most notably exhaustive testing evaluation. That is the way to reveal vulnerabilities and allow for the improvement of security assurance methods.
TechnicalTest-bed role in designation and implementation phasesSAFERtecR+I 7.3 Test Data Exchange Framework• A test-bed that emulates the full end-to-end (i.e., from the vehicle OBU to the RSU and cloud-services) automotive environment has been designed and implemented. Such test-beds are needed both in testing security assurance but also to serve any automotive testing purpose. They are expected to increase the engineering quality of automotive products if a number of technical issues (e.g., V2X messages incompatibility, security of V2X content etc) can be efficiently addressed.
TechnicalUnderstanding traffic users’ behaviour in complex scenarios ikey to establish more effective technological safety measures SIMUSAFER+I 2.5 Addressing User-Centric Development of CCAMSIMUSAFE infographic• The project contributes to understand the individual and collective behaviour of actors involved (drivers, two wheelers, pedestrians) and their interaction between themselves and safety-related systems and services.
• One of the project’s expected outcomes is understanding the usefulness of vehicle safety devices and the safer integration of new types of vehicles, i.e. automated vehicles, on the roads.
TechnicalChallenges in sharing trajectories for manoeuvres coordination TransAIDR+I 4.1 Physical and Digital Infrastructure (PDI)

R+I 2.1 Environment perception technologies for CCAM

R+I 4.2 Connectivity and Cooperative Systems
D6.2 Assessment of traffic management procedures in Transition Areas •Some services require the coordination of cooperative manoeuvres for CAVs. •This can be done both locally by the coordination between the affected vehicles, and they can be assisted by RSUs taking advantage of its inherently larger perception of the environmental scope.
•To allow the coordination between vehicles, it is necessary that they periodically transmit their future trajectories, so that other vehicles can compare their own trajectories with the received ones and predict potential problematic situations that can be avoided through cooperative manoeuvring.
•This, however, is a highly time-critical issue. Automated vehicles plan a spacious set of trajectories within milliseconds for the next discrete time frame to determine their next step.
•From an automation point of view, it might be nearly impossible to transmit one certain trajectory (for a larger time frame of seconds) with a confidence level high enough so that other vehicles can take it into account.
TechnicalImportance of realistic simulation of V2X communicationTransAIDR+I 4.1 Physical and Digital Infrastructure (PDI)

R+I 4.2 Connectivity and Cooperative Systems

R+I 4.3 Fleet and traffic management in a CCAM eco-system
D6.2 Assessment of traffic management procedures in Transition Areas The performance evaluation of the considered scenarios and parameter combinations has shown the following, which held true in both the first and second iterations:
•The realistic simulation of V2X communication has an impact on traffic scenarios, which makes them indispensable for a realistic performance evaluation of V2X traffic scenarios.
•Traffic management algorithms need to account for sporadic packet loss of various message types in some way.
•Although important, the realistic modelling and simulation of V2X communication also induces a significant computational overhead. Thus, from a general perspective, a trade-off between computation time and degree of realism should be considered.
TechnicalImproving the overall validity of simulation analysis TransAIDR+I 4.1 Physical and Digital Infrastructure (PDI)D8.2 Results of the meta-analysis (Chapter 4 lessons learned) •Longitudinal motion of CAVs was explicitly simulated on the tactical level since ACC/CACC systems dictate the CAV actions (acceleration/deceleration) in the next time step according to the leading vehicle behaviour.
•However, simulation of a motion planning algorithm for CAVs would provide additional testing possibilities in the context of the TransAID simulation experiments, such as cooperative manoeuvring with the use of realistic communication protocols (desired/planned manoeuvres).
•Furthermore, decentralised cooperative lane changing was integrated in the traffic management plans of specific use cases and evaluated based on safety, traffic efficiency and environmental KPIs.
•Similar work for centralised cooperative lane changing would enable the comparison of the two approaches and provide insights in terms of theirs advantages and limitations.
•Ultimately, evaluation of CAV vehicle-driver models and mixed traffic conditions in the context of simulation experiments that considered real-world road facilities and traffic conditions would enhance the overall validity of the simulation analysis.
TechnicalInfrastructure advice generation – tests required for each scenario TransAIDR+I 4.3 Fleet and traffic management in a CCAM eco-systemD8.2 Results of the meta-analysis (Chapter 4 lessons learned) •The advice generation can only be as good as the sensors and sensor fusion allows it to be.
•In addition, it is highly dependent on the precision of the traffic management algorithm and its implementation.
•The better the data and data fusion model are, the better the result.
•Nonetheless, the precision of individual advice may not be sufficient for CAVs due to various reasons, such as different behaviours of CAVs, integration with CAVs, or the advice compliance rate of CAVs.
•Because the confidence level towards the infrastructure advice plays an important role in when vehicles consider the advice, thorough tests should be performed in the future for every scenario.
TechnicalInfrastructure sensors and sensor data fusion TransAIDR+I 4.1 Physical and Digital Infrastructure (PDI)
R+I 4.2 Connectivity and Cooperative Systems
D8.2 Results of the meta-analysis (Chapter 4 lessons learned) Infrastructure should require technology to perform sensor data fusion:
•In the project, data was retrieved from multiple sets of induction loops as well as cameras.
•Mono cameras have been sufficient, but the usage of cameras is restricted, as GDPR must be applied.
•GDPR can result in down-sampling of image resolutions or online removal of faces/license plates.
•The fusion with V2X data is required to match received messages with local sensor data, which allows to specify which vehicle is being advised.
•Only with the augmented sensor data inputs, the infrastructure traffic management system can calculate (individual) advice for vehicles in the transition areas.
•For ideal advice generation, detection and continuous tracking of objects is required.
•To avoid latencies, the whole process of image generation, GDPR application, fusion, detection, and tracking needs to be fast, requiring high computer powers.
TechnicalInsights in OEM developments helpful for feasibility assessment of measuresTransAIDR+I 4.1 Physical and Digital Infrastructure (PDI)
R+I 4.2 Connectivity and Cooperative Systems
R+I 4.3 Fleet and traffic management in a CCAM eco-system
D8.2 Results of the meta-analysis (Chapter 4 lessons learned) •In terms of feasibility assessment, it would be helpful to get more insights into OEM developments.
•In this perspective, TransAID was happening very early, before several details of future (C)AV behaviour is known.
•The topic of ToCs, MRMs and transition areas should also be more in focus of larger scale projects, which would allow broader testing, but also standardisation of countermeasures and the identification of further required additional message parts, like e.g., sharing of (C)AV capabilities.
TechnicalManoeuvre coordination – additional work required TransAIDR+I 4.2 Connectivity and Cooperative SystemsD8.2 Results of the meta-analysis (Chapter 4 lessons learned) More work will be needed on manoeuvre coordination, given it is still at its early stages:
•In TransAID we have provided the basis for the decentralised and centralised coordination of vehicles, with the design of V2X message flows and generation rules.
•The obtained results show that reliable and accurate manoeuvre coordination is possible, but the channel load will need to be controlled effectively.
•Manoeuvre coordination requires the timely and reliable exchange of V2X messages, but current congestion control protocols may limit the transmission frequency of MCMs. This could thus affect the coordination time and further work will be required on this topic.
TechnicalV2X implementation for a more robust traffic managementTransAIDR+I 4.2 Connectivity and Cooperative SystemsD8.2 Results of the meta-analysis (Chapter 4 lessons learned) •The frequency of retransmission of the infrastructure advices should be further studied to guarantee the correct reception of the advices while keeping the channel load as low as possible to avoid negatively impacting the performance of V2X communications.
•Techniques that assure the correct reception of infrastructure advices should be implemented for a more robust traffic management tolerant to sporadic communications failures.
•Mechanisms guaranteeing the correct reception of infrastructure advices (such as acknowledgement communication packets (ACKs)) should be implemented for a more robust traffic management (as already discussed above).
•Synchronous packet transmission by all RSUs assumes they are out of interference range requiring adequate RSU placement and/or the addition of a random back-off mechanism to reduce the interferences between RSUs
TechnicalIntegrated Urban Mobility Pilots Transforming TransportR+I 5.5 System architecture for data sharingD9.5 – Impact of Big Data Technology on Integrated Urban Mobility Integrated Urban Mobility Pilots:
• Data Analytics: As main lesson learned with the issue of accuracy of GPS data point out that, to know the real/reliable behaviour of logistic vehicles in the city centre, more data sources are needed.
• Data Protection, Engineering, Standards: a general advice for future developments involved Data Analytics tasks and dashboard developments: use as much as possible open source technologies to process and analyse data, as well as in dashboard developments.
TechnicalSmart HighwaysTransforming TransportR+I 5.5 System architecture for data sharingD5.5 – Impact of Big data Technology for connected vehicles Smart Highways:
• Data Visualisation and User Interaction: the complexity of the dashboard has been, in its usability, a bit higher than expected. This could be improved through making the graphic interface at both levels, visualization and interaction. Secondary, although powerful to monitor in real time, it was pointed out by the Traffic Analysis Team that gathering raw figures was a must to be able to check theoretical error ranges (RMSE and R2).
• Data Analytics: there might be not enough data at all required clusters (e.g. day/time clusters: cluster 1. Woking Monday to Thursday, cluster 2. Working Friday), so grouping clusters is a necessary work although never optimal.
• Data Quality: For some of the datasets considered during project, it has been evident that not always the quality of the data was enough to be directly processed. A Data Quality Management Plan and a calendar of Quality Control tasks would be a must for us in next project.
• Data cleansing and conditioning has been carried out by the data analytics team, taking a high percentage of their effort in the project just for these activities, estimated in 80% of total work of Data Analytics Team.
TechnicalSustainable Vehicle Fleets Transforming TransportR+I 5.5 System architecture for data sharingD5.5 – Impact of Big data Technology for connected vehicles Sustainable Vehicle Fleets:
• Data Analytics: it was found useful to keep historical non reproducible data and, when possible, in raw format
• Data Processing Architectures: changes in the code and are relatively frequent and should be implemented and put into service as soon as possible. Assuring a Continuous Integration and continuous deployment implies an organizational architecture to which the use of microservices can help.
• Data Management: in order to provide services at real-time, extra storage is required (what it should be considered in the dimensioning phase of the system). It was also found that non-relational databases are more appropriate than classic relational databases, for evolving systems. Relational databases are more restrictive in their structure and do not allow rapid changes.
TechnicalTrustVehicle• The project scope is very wide. From cleaning of sensors to driver state monitoring technology to development of very detailed graphical user interfaces.
• The evaluation of the prototypes seemed to mostly been made based on powerpoint presentations in classroom settings and not very much in the field.
• HMI focus seemed to have been on developing rather basic graphical user interfaces which is better done at OEMs with proper UX departments. It would have been better to focus the evaluation more on the actual interplay between the human and the vehicle in the specific usecase where the UI would be just one small part. Test design principles rather than try to produce the detailed design.
TechnicalUpDriveD10.3 Periodic Technical Report• Resources for cloud stack maintenance and the ongoing technical support for a diversely skilled user base should not be underestimated
TechnicalUpDrive• Visual information has its limits in accuracy for drastic appearance changes. Fusing3D LiDAR information will allow to overcome them.
TechnicalUpDrive• Assuming natural scenes without any augmenting objects, it is easier to achieve good accuracy of rotation angles in on-line cross-calibration than to achieve reasonable accuracy for translations.
scene augmentation for close objects would result in better position estimates but this hypothesis has not been systematically tested
• Visual information has its limits in accuracy for drastic appearance changes. Fusing3D LiDAR information will allow to overcome them
TechnicalUpDriveD10.3 Periodic Technical Report• Sampling based motion planning good technique to capture complexity of urban traffic
• Interface between object prediction and motion planning relying on explicit predicted trajectories not in all cases useful
TechnicalDigital Twin – XiL testing vi-DASR+I 1.1 Pilots• The development of early integration platform for testing, covering from simulation to real testing plays an important role for complex projects including the integration of multiple sensors, modules and functionalities. The possibility of an early integration and running first test in virtual environments helps in the early detection of technical problems.
Technical Importance of improving connectivityMAVENR+I 4.2 Connectivity and Cooperative SystemsDeliverable D8.6 White Paper - Management of connected automated vehicles in a smart city environment• It is clear that digital connectivity is the underpinning infrastructure required to enable any ‘Connected’ application that forms the ‘Connected’ part of CAVs: •Fibre and V2X are the base level technologies required to deliver both the bandwidth and latency required for robust management of CAVs.
•Cities, and the private sector, are at varying levels of coverage for fibre across Europe and V2X is currently an evolving technology.
•Cities will need to continue to work with infrastructure providers to ensure greater levels of connectivity.
•Cities are, however, more progressed in deploying V2X technologies that enable communications with, and the potential management of, CAVs.
Technical
Legal/Regulation
New radio channels and changes in ITS communication architecture will be needed to cope with the high variety and amount of messages.TransAIDR+I 4.2 Connectivity and Cooperative Systems•Future deployment of V2X systems to support advanced traffic management services will require a variety of messages to be exchanged with low latency and high reliability.
•In TransAID different mechanisms to improve the reliability and scalability of the V2X message exchange have been proposed.
•However, to cope with the transmission of Day-1, Day-2 and beyond V2X messages, multiple radio channels will be needed.
•The standardisation work to adapt current V2X communications and networking protocols has just started within ETSI through the STF 585.
•To enable the use of multiple channels, changes in the ITS communications architecture as well as in the Facilities, Network and Transport, and Access layers will be needed.
Technical
Operational
Importance of RSU placement and connection to TMC TransAIDR+I 4.3 Fleet and traffic management in a CCAM eco-system
R+I 4.1 Physical and Digital Infrastructure (PDI)
R+I 4.2 Connectivity and Cooperative Systems
D6.2 Assessment of traffic management procedures in Transition Areas (Chapter 5 – Recommendations) •Infrastructure models are inherently part of the simulations of the use cases considered in TransAID.
•Geographical parameters, e.g., lane lengths, lane drop locations, merge areas, and placement of RSUs all contribute to the results of the use cases’ simulations.
•Furthermore, the setup of the infrastructure models is intertwined with the particular setup of AV behaviour models, traffic management algorithms, and communication protocols within these use cases and will, therefore, impact the results of the simulations.
•Consequently, the same attention should be paid to the real-world road-side infrastructure.
•In particular, experience with the use cases has shown that RSU placement and their type of connection to the central TMC should be taken into account.
Technical
Organisational/operation
Mediator• So far a lesson learned is to at an early phase have a clear view on the UC, the instrumentation of the demonstrators with a clear architecture and connection to the UC, and were they will be evaluated.
Technical
Organisational/Operational
Increase the time for the preparation of the demonstrators ADAS&MEFinal report ADAS&ME (D10.4) • As always there is important to not underestimate the time for the preparation of the demonstrations to be used during evaluations.
• A process of demonstrator technical integration is fully completed well in time before the commencement of evaluations is possible.
• Recommended time before evaluations are at least more than 2 weeks before to ensure that any late problems can be responded to and would give a better opportunity to consciously react and adapt the evaluation protocol accordingly.
• Perhaps most significantly this would minimize any impact on the schedule for final preparations.
Technical
Organisational/Operational
Integration in demonstratorsADAS&MEFinal report ADAS&ME (D10.4) • Review of physical and communication architecture is mandatory before integration starts, from an incremental integration approach (components-> systems-> verification, etc)
• Continuous risk assessment at all key phases of the project and at all levels; technical, operational, behavioral and legal.
• Stepwise and cross-use case integration approach (monitoring tool and plug fests) is a pre-requisite for a complex integration such as the ADAS&ME project.
Technical
Organisational/Operational
Importance of further developing traffic management for CCAM systemsMAVENR+I 4.3 Fleet and traffic management in a CCAM eco-systemDeliverable D8.6 White Paper - Management of connected automated vehicles in a smart city environmentTraffic Management is the least developed area of competence for the studied cities:
•While this is not an issue at present, as no fully automated vehicles are operating on European roads, this is an area of significant opportunity for development.
•Cities, their outsourced partners and industry, including MaaS operators, will need to further collaborate on future traffic management capabilities and how traffic management technologies and practices will need to be transformed and developed to allow for the implementation of policies emerging in this area.
Technical (Social)Complexity of traffic interaction to be considered interACT • Interaction of human traffic participant is very complex (see results from observational studies). The integration of Automated Vehicles needs to address this complexity.
• What we need in future for safe integration of automated vehicles in mixed traffic environments:
• Good understanding of interaction behavior; enhanced sensor algorithm capability; Fall-back layer for safety critical situations; inclusion of information from infrastructure and additional technology such as use of mobile phone data for better prediction of environment; cross-cultural studies; standardization of interaction strategies/HMI; evaluation of learning effects when interacting with Automated Vehicles.
Technical, evaluation L3PilotR+I 6.2 Socio-economic and environmental impact analysis and target-based assessment of CCAM benefitsD3.4 Evaluation Plan
Chapter 6 Lessons learned p. 107
• A special concern and source of uncertainty in AD pilots involves the potential differences between piloted ADF prototypes and their eventual market-ready versions. These differences may influence the assessment of broader social impacts and the scaling-up of pilot results, and must therefore be considered. L3Pilot resolved this issue by developing a description of mature function and its ODD togethether with ADF developers and utilised these descriptions in the impact assessment work and for scenarios used in socio-economic analyses.

Technical, evaluationConsideration of ADAS functions to include in the baseline conditionL3PilotR+I 6.2 Socio-economic and environmental impact analysis and target-based assessment of CCAM benefitsD3.4 Evaluation Plan
Chapter 6 Lessons learned p. 107
• Selection of a good baseline condition to compare impacts of ADFs is not straightforward, as a proportion of the existing vehicle fleet already features some automated functions (e.g. ACC). Therefore, baseline conditions should consider the penetration of existing levels of automation. This sets the challenge of understanding all types of automated functions included in evaluation scenarios and their interactions.
Technical, evaluation, human factors, evaluationUniformity in specification
Proofed interoperability
Proofed security of sending and receiving messages
User acceptance
Impact assessment
InterCor• A common set of upgraded specifications for ITS-G5 has been delivered;
• A common set of upgraded specifications for hybrid communication has been delivered;
• A common set of upgraded specifications for PKI has been delivered
• A common set of upgraded specifications for services has been delivered
• Four interoperability tests have been organised in the four InterCor MS to validate the common specifications
• Pilots based on the common specifications have been carried out/are running in the four InterCor Ms
• Roll-out guidelines and best practices for pilot operation have been published
• Management processes must remain flexible and responsive to project needs
• It is importnat to agree a Framework that is useful to evaluation experts whilst being understood by the broader staekholder base
• Embedding evaluation in the project as the outset is imperative
• The need to focus on a limited number of incisive, common and achievable evaluations is important to achieving a successful outcome​
• TESTFESTs provided an opportunity for a qualitative verification, but are not sufficiently accurate to verify interoperability and implementation of specification
• Having high quality and reliable services is key to user acceptance​
• Human Machine Interface (HMI) design and position mounting influences user acceptance – better integration into vehicle is desirable (e.g. via Android Auto, Apple Car Play) to reduce driver distraction improve observation of warnings (e.g. option for audible warning useful)
• Seeing the difference in the raw data is difficult to measure. Ideally a larger pilot with measured baseline and at least 20% of drivers (min penetration) enabled with C-ITS to be considered​
• Using a road with little or no existing ITS services to avoid direct comparisons with existing ITS services would make ‘HMI reaction’ easier to measure​





AutoMate• Development of new, complex systems such as components to enable self-driving cars cannot be planned in detail over a long period. The key to success is quickly gaining insight into what is possible and desirable, and adapting direction and pace if required. Some call that agile development. The nature of EU projects runs counter to that requirement. The project must be fully planned 3 years ahead. Lessons learned within the project usually cannot be immediately used to adjust the course of the project.
• Overly complex developments are bound to fail. Focus on a particular issue is key. A complete self driving car has many domains such as perception, control algorithms, routing,  HMI etc. Trying to address many of these domains as was done in AutoMate within a research project will lead to isolated small developments in each domain, because the required integration at the end of the project is extraordinary challenging, and presupposes a development stage within the domains which cannot be guaranteed within the project.
• Knowledge transfer or retention after projects involving industrial partners is very limited. Understandably, these partners have commercial interests they will protect. This can run into conflict with the desire of the academic research community to share insights such as data, algorithms or software. For others, to profit from these types of experience means they need to “get their hands on it”, because high level descriptions are usually not adequate to retrace the steps taken by the project. This can run counter to industrial partners’ exploitation plans, and thus creates competing interests.

Feedback form

Have feedback on this section? Let us know!

Send feedback

Feedback

Please add your feedback in the field below.

Your feedback has been sent!
Thank you for your input.

An error occured...
Please try again later.