2021 Gaps and Recommendations

Introduction

In the last years, a big effort has been dedicated by many companies, associations, research institutes and universities in standardisation of Connected and Automated Driving (CAD) technologies.

This article aims to collect and investigate the main gaps in CAD standardization. The process of collection has been performed by searching for public technical reports, and by discussing with several experts working in academia, in the automotive industry, in research institutes, and in standardisation bodies.  Indeed, this is a hard task because the topic of connected and automated driving is wide and it requires a deep knowledge on different fields. This article attempts to summarize and harmonize all the gaps and recommendations related to CAD standards emerged from different source of information.

In order to help the readers, all recommendations are marked in bold throughout the text and summarised in the final sections.

Data analysis

Since 2015, the number of published standards per year increased exponentially (figure 1) demonstrating a an increasing needs for standardisation. Until August 2021, 175 standards in the field of connected and automated driving have been published, and 51 standards are still under development. An impressive number of standards have been published so far. However, there are still some gaps to be covered in the next years

Figure 1: Number of standards per year related to connected and automated driving vehicles

On the web page Standard Collection, all the CAD standards are listed and categorized. They have been distinguished in two categories: under development and published. Furthermore, each standard has been assigned to the following domains:

• Terms & Definitions
• Management/ Engineering Standards
• AD/ADAS functions
• Testing, Verification & Validation
• In-Vehicle Systems, Networks, Data and Interface Definition
• Connectivity
• Human Interaction
• Artificial Intelligence
• Safety
• Privacy & Security
• Map and positioning

Note that the domains have been chosen before starting the standard collection.

In the figure 2 are graphed the number standards per each domain, divided into published and under development.  From the graph, three indications appear clearly:

1. Connectivity is the area with the highest number of standards. This result is completely expected because no communication can be established without a standardisation of interfaces, bandwidths, frequencies, protocols, etc.
2. Testing, Verification & Validation is the domain after Connectivity which has the greater number of standards. Firstly, this mostly thanks to the work of the Association for Standardization of Automation and Measuring Systems (ASAM), who is steering and harmonizing all standards related to simulation and virtual validation [7]. Since 2018, OpenDRIVE, OpenScenario, OpenCRG, Open Simulation Interface (OSI) are managed by the ASAM. Secondly, validation of CAD systems represents one of the bottleneck for their commercial deployment. Therefore, an increasing effort is dedicated on standardization and regulations [8] of validation methods and methodologies.
3. Artificial Intelligence (AI) is the only domain where no standards have been found, unless was expected.

In particular, the lack of AI standards suggested that further investigation were needed. In the following section, the topic of AI has been analysed.

Figure 2: Connected and automated driving standards divided by domains

Standards for Artificial Intelligence

From a further investigation of AI standards, it emerged that the International Standard Organization (ISO) and International Electrotechnical Commission (IEC) decided in 2018 to create the subcommittee (SC42) dedicated to the topic of the Artificial Intelligence. the subcommittee SC42  is part of the Joint Technical Committee “Information Technology” (JTC1) . The subcommittee SC 42 is divided into four working groups:

• WG1: Foundation standards
• WG2: Data
• WG3: Trustworthiness
• WG4: Use cases and applications

All the standards in preparation by these working groups are not specifically developed for connected and autonomous driving field, but they take into account a wide range of applications.

In the automotive industry the standardization activities related to functional safety (ISO 26262) and SOTIF (Safety of the Intended Functionality, ISO 21448) have started to consider the problem of introduction of artificial intelligence algorithms in the ADAS and autonomous driving systems.  However, to the best of our knowledge the safety problem related to deployment of AI software in automotive system products is still open.

Nevertheless, the work accomplished inside the WG3 could potentially help to improve the safety of connected and automated vehicles.  In particular, the WG3, which is working on trustworthiness, has the following tasks [4]:

1. Establish trust in AI trough transparency, verifiability , explicability and controllability.
2. Investigate threats and risks of AI systems
3. Investigate approaches to achieve AI systems robustness, resiliency, accuracy, safety, security privacy.

Firstly, the concept of transparency and explicability is extremely important in those software modules responsible for the decision and execution of the automated manoeuvres. In case of an accident due to an apparent wrong manoeuvre, the decision process followed by the autonomous driving system before the accident event must be completely transparent and explicable.

Secondly, the topic of threats and risks is essential to assess ethics and goodness of choices. In particular, considering the nature of some AI techniques, the goodness of choices needs to be assessed by using a statistical approach. Indeed, risks cannot be eliminate, but it could be reduced as much as possible. The remaining residual risk needs to be quantified to understand if could be socially and statistically accepted.

Finally,  the topic of robustness is very important for deploying AI models in CAD vehicles. The word “robustness” is used as a general terms for describing a series of properties of AI techniques [4] [5]. However, we can summarize the robustness as the property to maintain unchanged the performance with respect to small input variations. In detail, it is not required that the outputs remain exactly the same with respect to small input variations, but the overall performance has to remain unchanged. As an example, considering to have an artificial neural network able to recognize cars with a detection rate of 90%. The property of robustness could require that the detection rate remains unchanged regardless of time of day.

In order to verify the robustness and performance of AI models, formal methods cannot be used in practice, therefore statistical methods have to be implemented. On this regards, as suggested in [4], it is needed an international standard for field testing. Probably, this could vary significantly between industries and type of applications.  Regardless of the industries, the field tests can increase the trustworthiness and the safety, or in general the credibility of this techniques. Moreover, this standard could help to define responsibilities of different stakeholders during design, implementation, verification and certification phases.

As mentioned before, all these aspects are crucial for AI deployment in CAD vehicles. For this reason, the WG3 is proceeding in the right direction to provide solutions to these challenges and enable automotive industries to use safe AI technologies.   

However, liaison activities are needed between JTC1/SC42 and the other technical committee (TC) :

• ISO/TC 204 Intelligent Transportation System
• ISO/TC 22 Road Vehicle
• IEEE P2846 Autonomous Vehicle Decision Making

Standard recommendations in 2017-2021

The scope of this article is to give to readers a series of recommendations on CAD standardisation . However, it is important to mention that the authors of this article are not the only ones trying to analyse the standards published so far and to highlight the main gaps. Four organisations worked on the same topic in the last years. They published the following five documents:

1. ISO/TR 20545:2017 “Intelligent transport systems — Vehicle/roadway warning and control systems — Report on standardisation  for vehicle automated driving systems (RoVAS)/Beyond driver assistance systems”, July 2017
2. IEEE P2020 – Automotive Imaging White Paper, 2018
3. “Standardisation  Roadmap for Automated Driving”, Verband der Automobilindustrie (VDA), May 2019
4. “Connected and Automated Vehicle Technologies – Insights for Codes and Standards in Canada”, Canadian Standardization Association Group (CSA), June 2020
5. ISO Report on standardisation prospective for automated driving (RoSPAV), January 2021

In particular, the last three publications have been published recently, thus they can offer valuable information to understand the remaining gaps.

ISO/TR 20545:2017

This technical report published in 2017 offers some interesting recommendations, some of them have been already included in new standards. However, some others are still not considered so far. In this article the remaining standard gaps are emphasized.

The ISO/TR 20545:2017 advises to standardise functional requirements, functional architecture and interfaces between the different components. This standardisation will help the industry to be faster and it will increase the compatibility between different solutions in the market. Until now, there is only the standard ISO 23150:2021 that is proceeding in this direction trying to standardize the interface between perception sensors and data fusion software modules.

Furthermore, the ISO technical report recommends the standardisation of requirements of some particular safety critical modules:

• Requirements between the driver and system at each level of automation and during the control transition
• Requirements of system operations in case of a system malfunction.
• Requirements for reliability and certifications.

IEEE P2020

The IEEE P2020 Working Group was created in 2016 during the AutoSense Conference in Brussels [9]. This group aims at defining a common approach in the automotive industry to measure image quality. The working group is aware that vehicle manufacturers, tier 1 and tier 2 lack of a common language for describing the quality of images. This ambiguity has created tensions between stakeholders during project executions and it has raised the project costs.

Currently, there is already a standard (IEEE std 1858) for mobile phone quality image, however it is not applicable to automotive use cases, which are more challenging.  In automotive, the landscape of imaging conditions is varied and unique (fish eye, multi-camera, high dynamic range, etc). Moreover,  the automotive camera system has to deal with challenging environmental conditions such as a wide range of weather,  illumination and temperature conditions. Considering the current gaps, the working group IEEE P2020 has the goal to define standards that include relevant metrics and key performance indicators for automotive image quality. This will benefit all the actors of automotive supply chain, helping them to define appropriate requirements, to measure the performance and to communicate properly the expected image quality.

More detailed information related to the automotive image quality gaps could be found in [9].

VDA recommendations

The technical report on standardisation roadmap by VDA has been published in May 2019. Since then, a lot of standards have been published or are under development. After reviewing the contents of the VDA report [2], there are some important domains, which still requiring a standardisation. The recommendations have been summarized and categorized into domains:

Domain	Recommendations by VDA
Management/ Engineering Standards	Inspection requirements for operators managing autonomous vehicle fleet. The suitability of this operator has to be confirmed in the future by certifications. Requirement of operational management of automated vehicle fleet during the phase of parking, service, refuelling, access authorization and fault management.
Safety	Ethical requirements on the actions to undertake in case of an emergency situations. A social recognized model to compare the performance of “safe automated driving functions” with respect a human driving performance. This is very useful for CAD acceptance and introduction
AD/ADAS functions	Recognition of emergency vehicles belongs to police, firefighter, ambulances, etc.

CSA Group recommendations

The CSA published an interesting report in June 2020 [3] based on a collaborative work of 14 stakeholders belonging to automotive industry (OEM, technology companies), public transportation agencies, academic and research institutions and standard developing organizations. In this report CSA has highlighted gaps and needs in regulations and standards. All recommendations have been divided in 8 themes and summarized in the table below:

Theme	Recommendations by CSA
Harmonization and interoperability	Harmonization of standards dedicated to V2X (vehicle-to-everything) communication globally and North America as a priority. In particular, on band allocation. Standardisation  of communication in emergency situations and for standardisation  of information related to road condition and weather.
Uncertainties with enabling communication technologies	Make a clear decision on type of technology to use for the communication: DSRC or C-V2X. Lack of standards in long range wide-area applications
Compliance verification	Define responsibilities for certification of CAD technologies. It could be OEM and Tier1 responsible for the certification  (or better self-certification) or a third party entity.
Physical infrastructure	The question of whether CAD vehicles have to adapt to infrastructure or vice versa was raised, and it was noted that there is a benefit to develop physical infrastructure guidance that can inform infrastructure owners and operators (IOOs) as it relates to line markings, static signs, electronic signs, road works, road continuity.
Operation Design Domain	Develop standards for certifying roadways to what level of automation they can support. It is fundamental to define the measurable conditions (temperature, precipitation, etc.), the type of road, the marking signs, the connectivity status that allow or not allow a CAD vehicle to operate.   Define standards focusing on adaptive vehicle level of automation for different user needs and conditions.
HD mapping and localization	Standards on map creation will help increase compatibility between different providers.
Cybersecurity and Protection of privacy	There are already a lot of activities in cybersecurity space. However, there are some concerns related to SCMS (Security Credential Management System), that manage credentials and ensures secure V2X. One is related to misbehaviour detection management and the other on privacy.
Technology Maturity	In this area are identified gaps related primarily to edge cases and to difficulty to identify objects and situations under non-optimal conditions, e.g. objects with a low reflectivity,  less common objects (like motorcycles), and construction zones.    Connectivity is a viable solution in many cases by providing additional information to the vehicle.  At the same time, it is assumed that as more demonstrable experience is attained, there will also be a natural enhancement of technical capabilities through improved machine learning.

ISO Report on standardisation prospective for automated driving (RoSPAV)

In January 2021, the ISO has been published a technical report summarizing the current projects and standards, and the future standard activities. This report offers also a good overview of ISO subcommittee (SC) and ISO working groups (WG) involved in the CAD standards. TC 22 Road vehicles and TC 204 Intelligent transport system are the most active on this field.

In 2017, the ISO/TC 22 Road vehicles created an ad hoc group ADAG (Automated Driving Ad hoc Group) in order to propose a road map for the standardisation of CAD technologies. The scope of this group was to:

• Identify gaps, redundancies, and opportunities
• Promote collaboration between the different WGs belong to TC 22
• Promote discussions between different TCs. In particular with TC 204 Intelligent transport system and ISO/IEC JTC1.
• Explore the feasibility of developing global goals in collaboration with other standardization bodies, such as Society of Automotive Engineers (SAE).

The main results of the ADAG are summarized hereafter.

Theme	Recommendations by ISO – ADAG
Minimal Risk manoeuvre	Standards for defining a minimal risk manoeuvre of electrical vehicle. The design of an electrical vehicle has to consider the battery disposal for performing a minimal risk manoeuvre.
Driving monitoring system (DMS)	Standardisation of different levels DMS and feature, which has to take into account the levels of automation, the driver’s role, the use cases, the ODD, the customer personalization.
Internal and external HMI	Standardised HMI internal messages for communicating: – The level of automation – The use cases – The ODD – The take-over request   Standardised HMI external messages that communicate to road users the intention of connected and autonomous vehicles
Perception	Standard methodology assessment of the perception performance
Data storage	Standard on the data storage system for automated driving (DSSAS). This is a complementary system to the Event Data Recorder. The DSSAS provides information about take-over request, driving delegation activation, minimal risk manoeuvre performance.
Validation	Standardize the methodology and the tools capable of different validation activities to demonstrate the validation completeness.
Digital mapping	Harmonise standards related to digital mapping systems
Infrastructure	Global standard for horizontal and vertical signs.

Unfortunately, the ADAG group is no longer active. However, considering the transversal competencies required to design, build and validate connected and autonomous vehicles, the TC 22 decided to continue the coordination activity through a permanent group composed by each representative of the SCs involved. The new group is co-chaired by one representative of the TC22 and one representative of the TC204. This new group is named  ACDG (Automated Driving Coordination Group). It is principally involved in this cluster:

• Cluster A “Safety”
• Cluster B “Security”
• Cluster C “HMI”
• Cluster D  “AI/ML” Artificial Intelligence and Machine Learning for automated driving

Final recommendations

The landscape of standardisation of CAD vehicles is very wide and around 200 standards have been published in different domains so far.

This article attempts to give recommendations based on the investigations performed by other consortium and on expert judgements of professionals working in different fields. Final and harmonised recommendations are provided in the table below. They have been categorized  into domains to facilitate future discussions.

Indeed, further collaborations are needed to promote an open discussion between experts on each single domain and representing the interests of different parties (OEM, Tier 1, certification entities, academia, national and international authorities, etc.)

Domain	Final Recommendations
Terms & Definitions	None
Management/ Engineering Standards	Standard on inspection requirements for operators managing autonomous vehicle fleet. Standard on requirements of operational management of automated vehicle fleet.
AD/ADAS functions	Standardise functional requirements, functional architecture and interfaces between the different components. Standard on requirements of system operations in case of a system malfunction. Standard for the recognition of emergency vehicles belongs to police, fire-fighter, ambulances, etc. Define standards focusing on adaptive vehicle level of automation for different user needs and conditions. Standards for defining a minimal risk manoeuvre of electrical vehicle. Standards for certifying roadways to what level of automation they can support (ODD definition).
Testing, Verification & Validation	Standard on requirements for reliability and certifications. Standard methodology assessment of the perception performance. Standardize the methodology and the tools capable of different validation activities to demonstrate the validation completeness. Standard to compare simulation results with real tests. Standard for ontology definition to have an unified meaning of data independent from the underlying IT implementation. Standards that include relevant metrics and key performance indicators for automotive image quality.
In-Vehicle Systems, Networks, Data and Interface Definition	None
Connectivity	Harmonization of standards dedicated to V2X communication globally. This concerns not only frequency allocations, but also differences in the protocols and even cryptographic algorithms used in different markets, notably the US, Europe, and China. Standardisation  of communication in emergency situations and of information related to road condition and weather. Standards in long range wide-area applications.
Human Interaction	Standard on requirements between the driver and system at each level of automation and during the control transition. Standardisation of different levels DMS and feature, which has to take into account the levels of automation, the driver’s role, the use cases, the ODD, the customer personalization. Standardised HMI internal messages for communicating: the level of automation, the use cases, the ODD, the take-over request. Standardised HMI external messages that communicate to road users the intention of the connected and autonomous vehicles.
Artificial Intelligence	International standard for field testing Liaison activities are needed between JTC1/SC42 and the other technical committee (TC) : – ISO/TC 204 Intelligent Transportation System – ISO/TC 22 Road Vehicle – IEEE P2846 Autonomous Vehicle Decision Making.
Safety	Standardised social recognized model to compare the performance of “safe automated driving functions” with respect human driving performance. Standard on ethical requirements on the actions to undertake in case of an emergency situations.
Privacy & Security	None
Map and positioning	Harmonise standards related to digital mapping systems.
Infrastructure	Global standard for horizontal and vertical signs.

            Last update: August 2021

Bibliography

[1] “ISO/TR 20545:2017 Intelligent transport systems — Vehicle/roadway warning and control systems — Report on standardisation  for vehicle automated driving systems (RoVAS)/Beyond driver assistance systems”, ISO/TC 204, July 2017. Link

[2] “Standardisation  Roadmap for Automated Driving”, Verband der Automobilindustrie (VDA), May 2019. Link

[3] “Connected and Automated Vehicle Technologies – Insights for Codes and Standards in Canada”, CSA Group, June 2020. Link

[4] Thomas Zielke, “Is Artificial Intelligence ready for standardisation?”, Proceedings of 27th European Conference, EuroSPI 2020, Düsseldorf, Germany, September 9–11, 2020. Link

[5] “ISO/IEC TR 24029-1:2021 Artificial Intelligence (AI) — Assessment of the robustness of neural networks — Part 1: Overview”, ISO/IEC JTC1/SC42, March 2021. Link

[6] “ISO Report on standardisation prospective for automated driving (RoSPAV)”, January 2021. Link

[7] “ASAM SIM: Guide – Standardization for Highly Automated Driving”, ASAM. Link

[8] “New Assessment/Test Method for Automated Driving (NATM) Master Document”, UNECE, Validation Method for Automated Driving (VMAD), February 2021. Link.

[9] “IEEEP2020 Automotive Imaging White Paper”, Member of IEEE P2020 Working Group, 2018. Link.