As we enter an era increasingly populated by highly autonomous vehicles, there is a vast range of dynamic driving scenarios that Automated Driving Systems (ADS) may encounter. From hazardous environmental conditions to internal system failures and external cybersecurity risks, ensuring ADS safety across diverse operating situations is essential for enabling safe autonomous driving experiences.
The recent release of “ISO 34505: 2025” underscores this need by providing a structured framework for generating, evaluating and managing test scenarios that reflect real world driving conditions. By standardizing how test scenarios should be defined and tested, the initiative aims to enable consistent, repeatable validation practices across the industry and thereby support development of robust ADS provision.
As autonomous systems grow more complex, the need for robust, scalable validation practices become increasingly critical. In response, an integrated approach — combining regulatory audits, system-level testing and adversarial simulations — provides OEMs and Tier 1 suppliers a structured path for both vehicle safety and regulatory compliance. Focusing on cybersecurity, this blog outlines the key components and methodologies of ADS Validation, and demonstrates how an integrated approach can be effectively executed.
Automated Driving System (ADS) Validation: Approach & Methodology
According to “SAE J3016: 2021”, Autonomous Driving System (ADS) refer to the collective technology stack responsible for performing dynamic driving tasks (DDT) at SAE Level 3 and above. With the system taking full responsibility for autonomous decision-making and vehicle control, validating ADS safety calls for identifying diverse validation targets and a multidisciplinary process for executing them.
I. Approach
The UNECE WP.29 Working Group emphasizes ADS Validation should be approached from multiple angles, including audit and assessment, simulation and virtual testing, real-world testing and more. Drawing on key industry whitepapers (e.g. The Autonomous Working Group, Association for Standardization of Automation and Measuring Systems, Mercedes-Benz), validation efforts can be broadly categorized into three core pillars: functional performance, internal system reliability and external cybersecurity resilience.
The first pillar, Functional Performance, focuses on ensuring the embedded vehicle system behaves as expected across a full range of driving conditions — particularly under abnormal scenarios such as complex environments or sensor limitations. In alignment with the “ISO 34505: 2025” standard, which outlines scenario-based ADS testing, this pillar evaluates system capabilities in perception, decision making and control execution under realistic conditions.
The second pillar, Internal System Reliability, addresses resilience against system-level faults. This includes the inspection of fault detection mechanisms, hardware failure mitigation strategies, and adherence with Automotive Safety Integrity Level (ASIL) grades. Relevant to the “ISO 26262: 2018” standard defining the framework around electrical/electronic (E/E) system failures, this pillar assesses the system’s ability to maintain safety in the presence of internal malfunctions.
The third factor, External Cybersecurity Resilience, evaluates the system’s tolerance against external cybersecurity threats. Verification over secure communication and data integrity under potential attacks such as vehicle hacking, spoofing and denial-of-service (DoS)) is a key objective of this pillar. Associated with the “ISO/SAE 21434: 2021” standard illustrating cybersecurity risk management for vehicle E/E systems across the lifecycle, this phase assesses the system’s ability to proactively mitigate attack vectors targeting sensors, ECUs and OTA updates.
II. Techniques
While various techniques exist to evaluate functional performance, system reliability and external attack resilience, this blog focuses on three core cybersecurity validation methods — Compliance Auditing, Software-in-the-Loop (SiL) Module Testing, Hardware-in-the-Loop (HiL) Penetration Testing — to better illustrate the differences across diverse validation approaches.
The first technique, Compliance Auditing, focuses on verifying whether development practices and system architectures align with established safety and cybersecurity regulations (e.g. ISO/SAE 21434, UN R155). This method is widely used by OEMs and Tier 1 suppliers to conduct gap analyses during early-development stages or in preparation for CSMS Certification audits, to check whether internal processes conform to regulatory requirements.
AutoCrypt CSTP Compliance serves as a representative tool to accommodate these needs by validating vehicle vulnerabilities on a unified platform. It supports multiple testing domains including Security Validation, Functional Testing, Penetration Testing, Fuzz Testing and Vulnerability Testing and consolidates results into a comprehensive report suitable for regulatory submission. By combining testing execution and documentation, it reduces redundant tasks and streamlines the compliance process.
Another key validation technique is Software-in-the-Loop (SiL) Module Testing, which assesses robustness of embedded security components in virtualized test environments before hardware integration. Commonly applied to TEE (Trusted Execution Environment) based key management testing and V2X certificate handling simulation, this technique enables rapid iteration and early validation of security logic in controlled conditions, before advancing to high-cost hardware testing.
In accordance with these needs, the AutoCrypt CSTP Functional Tester validates hardware-dependent security functions using virtual ECU models in a Software-in-the-Loop (SiL) environment. By integrating communication interfaces, debugging tools, ECU source code and test code, this solution facilitates early detection of design flaws and integration issues well before mass production.
Another core testing approach is Hardware-in-the-Loop (HiL) Penetration Testing, which evaluates cybersecurity resilience of physical ECUs by simulating real-world attack vectors in controlled HiL testing environments. Often applied for in-vehicle network fuzz testing and Telematics Control Units (TCUs) penetration testing, this technique identifies system vulnerabilities under actual runtime configurations, moving beyond theoretical scenarios.
Serving this purpose, the AutoCrypt CSTP Fuzzer solution actively injects malformed, unexpected inputs into in-vehicle networks to test ECU-level resistance to cyber intrusions. Covering a broad spectrum of communication layers including the Network Layer (e.g. CAN, CAN-FD, Automotive Ethernet), Application Layer (e.g. UDSonCAN, UDSonCAN-FD) and Transport/Data Layer (e.g. VehicleCAN, VehicleCAN-FD), the tool enables precise testing of vehicle systems under a wide range of adversarial conditions.
Effective ADS Validation through an Integrated Approach
With a wide range of checkpoints to address and multiple techniques available, establishing a cohesive and effective strategy for ADS validation is essential. To meet this need, a structured progression — from Compliance Auditing to Software-in-the loop Testing and finally to Penetration Testing — offers a practical pathway for comprehensive and efficient ADS validation.
- At the first stage, Compliance Auditing defines the baseline and sets the strategic direction through regulatory compliance and process control.
- Next, software design implementation and testing activities are supported through Software-in-the-Loop (SiL) Module Testing, which enables validation before hardware integration.
- Lastly, Hardware-in-the-Loop (HiL) Penetration Testing technique can be utilized to observe real-world cybersecurity readiness under adversarial conditions.
This layered approach demonstrates how each phase builds upon and reinforces the next, enabling a robust and scalable validation framework.
With AUTOCRYPT being an authorized Vehicle Type Approval (VTA) Technical Service (TS) Provider , the firm is uniquely positioned to integrate diverse testing techniques and facilitate comprehensive ADS validation through the AutoCrypt CSTP Platform. From the AutoCrypt CSTP Compliance, which ensures design-level safety, to the AutoCrypt CTSP Functional Tester, which verifies correct functional behavior and the AutoCrypt CSTP Fuzzer able to test attack resilience, the platform enables a unified security analysis by consolidating all validation layers into a single, integrated platform.
Supporting a streamlined process for Vehicle Type Approval from ADS validation to export of results into compliance documents (e.g. TARA Report, Cybersecurity Test Report), the whole approval process can be effectively managed.
To learn more about the Autocrypt CSTP platform, check this page. For more information about our comprehensive suite of our automotive products & offerings, check this page.
