PDF
Siemens solutions foster continuous verification and validation
Key takeaways:
- Delivering reliable, safe autonomous and ADAS vehicles requires a data-driven, closed-loop development process and robust, continuous verification and validation processes focused on multiple levels from the chip to electronics, to the vehicle, and ultimately a city’s infrastructure (i.e., from chip to city).
- Only a fully integrated, complete lifecycle solution can move and sustain an automaker into a ADAS and AV transportation leader for tomorrow—a solution that fosters discovery to learn, adapt, and continually improve ADAS knowledge.
- Performance engineering must embrace continuous product evaluations, continually adjusting to customer usage patterns to provide safer operations.
- Trust in automation comes from constant performance awareness. The most trusted autonomous vehicle automaker will become the most successful.
- Siemens provides a robust set of solutions for managing the evolving usage patterns and applying them for virtual evaluations using technologies that make a comprehensive digital twin interactive, accelerating understanding and confidence of new ADAS/AV designs.
CIMdata’s definition of product lifecycle management (PLM) emphasizes the complete lifecycle of a product and all its related data, from requirements to actual performance, to planning and supply chain participation, then mass production, product service and upgrades, and finally decommissioning and recycling. Systems Engineering improves Autonomous Vehicle (AV) development and operations by considering a broader context—one that includes the operational environment of the vehicles.[1]
In a recent CIMdata eBook[2] covering AV development, CIMdata described the need for a broader view of collaborative engineering and operations where the AV may well rely on feedback from the city and other AVs’ sensors to provide safer, efficient route guidance, and robotic driving. In that eBook, CIMdata introduced the notion of continuous AV performance engineering where test sets can be used to perform verification and validation evaluations on demand, at different layers of complex AV systems.
Continuous Discovery Drives Constant Verification and Validation
Each loop within the AV ecosystem collects empirical measurements needed to train models making them trusted. Sensor and actuator models will be needed for each subsystem under test, providing stimulus and response such the subsystem performs as if it were in an actual driving scenario. With trusted models, teams collaborate faster making virtual verification and validation possible. When combined with city level abstraction, see Figure 1, and architectural frameworks,[3] an even broader ecosystem must be validated. This city/operational validation continues to evolve as more AV products are introduced into different city contexts. It is likely that AV operations will need to adapt to the city they operate in as expanded sensing is deployed at different times to improve traffic congestion management. With modern communications and connectedness,[4] continuous improvement of AVs during operations will occur. Validation of an upgrade must be managed, even when that upgrade is more frequent and likely asynchronous with city fleet management upgrades. Assuring safety by applying trusted, shared models that learn from expansive sensing of the vehicles’ operation is now possible.
Figure 1—System of Systems AV Product Development
(Courtesy of Siemens)
Building trust in virtual models of an operational environment requires performance engineers to conduct constant or continuous verification and validation. Automating the execution and evaluation of test suites as often as a new feature set is ready will become a competitive advantage over manual and open loop hardware in the loop (HIL) testing. Orchestrating testing across everything from sensing to engineers to traffic managers and their systems is needed for autonomous vehicles. Incremental enhancements to Advanced Driver Assistance Systems (ADAS) vehicles will be the basis to bridge the ADAS/AV gap.[5] Incremental changes are realized only when effective verification and validation are done incrementally. The management of test suites at these different levels, from city to vehicle to chip, need solutions appropriate to technology being evaluated. It seems certain coordination between the levels of testing, from component verification to subsystem and system validation would benefit AV development, resulting in shorter time to market. Performance engineers conduct product level validation tests as one of the final steps before a product is released for mass production. Performance engineering goes beyond verification test suites and includes an assessment of varying conditions like weather, velocities, road condition, friction, and many more. Select validation test cases must be repeated when a vehicle upgrade is made. The event triggering that validation test is different than the mass production launch.
AV development requires considering complex operational scenarios. This means building models at each layer of abstraction from diverse data silos with their insightful empirical data repositories. Leveraging real-world measurements to correlate virtual model-based scenarios using digital twins to help assess complex operational anomalies will make AV upgrades safer and more reliable, thereby fostering societal trust. Learning and adaptation keeps models trustworthy. Doing this in near real-time keeps them well correlated. Automated, continuous verification and validation are also required.
One last point, teams working collaboratively in parallel while using the same trustworthy models will develop enhancements and/or investigations from different expert contexts. Contexts for decision-making will be improved with automated, continuous verification and validation derived from actual usage patterns, ideally from real customer experiences. Performance engineering is becoming continuous for ADAS vehicles and AVs, performed whenever a new release is provided to the customer.
How Siemens Enables Continuous Verification and Validation
Siemens recognizes that all core business processes for all product disciplines must perform verification and validation, from components to final systems. Software systems with millions of lines of code rely on automated, daily test suites where anomalies are detected automatically. Quality teams then assess and often adjust the next release, which is provided in a few days, not years. AV development and operations need to expand beyond product engineering to include stakeholders outside of the automaker. Siemens has embraced the need for continuous performance engineering driven by faster product releases and upgrades. They have capabilities used across the AV chip to city abstraction layers depicted in Figure 1. By combining applications from their acquisitions over the past few years, Siemens has testing automation capabilities that span the major AV development domains. Figure 2 summarizes these domains and the broad tool suites: Prescan 360, EDA, and PAVE 360. Functions from systems engineering to systems interaction to verification and validation are all supported by managing simulation scenarios across the tool suites aiding development of all AV abstraction layers.
Figure 2—PAVE 360/EDA/PRESCAN 360 Provide Verification and Validation from Chip to City
(Courtesy of Siemens)
Prescan360 Benefits
Siemens Prescan 360 supports performance testing by synthesizing and running repeatable scenario tests with varying boundary conditions. By using models of sensors and vehicle dynamics, the signals a control system would monitor can be emulated making the control system react as it would in the final product. Performing these system evaluations before the commitment to hardware enables faster design cycles and builds confidence that the components will work together when integrated. AV adaptation to its operational environment needs Prescan 360 capabilities to manage the evolving scenarios as they are discovered, using trustworthy digital twins that accurately represent the systems they operate in (e.g., the city).
Scenario based testing requires empirical data that captures the movement of people, other vehicles, environment conditions, and the customers’ vehicles. These usage contexts and the best empirical data establish the appropriate digital twins. AV development needs these broader domains to provide different viewpoints (i.e., contexts) beyond any single vehicle. Once scenarios are captured, test automation provides virtual road tests simulating thousands of miles of use.
Automating scenario variations and executing simulation runs with post-processing into digestible reports is possible with Prescan 360. This makes affordable re-validation after mass production as AV or city-specific enhancements are realized. Continuous upgrades and taking advantage of external sensing for traffic, obstacles, and even time of day will make for safer AVs.
Choosing which scenarios to evaluate as a product’s features change needs grooming. A set of scenarios determined by performance engineers is a collection that must be managed as criteria when assessing each feature. Prescan 360 managed collections of scenarios can further optimize continuous verification and validation. This testing takes only a fraction of the time as physical driving tests would entail. For more in-depth technical description, please review Siemens’ Prescan 360 Data sheet.[6]
Electronic/Electrical Design Automation (EDA) Benefits
Siemens’ Integrated Electrical and Electronic Solutions[7] (aka EDA tools) provide an extensive solution set that support the development of electronics, embedded software, and allocation, and communications subsystems. Integrating this proven EDA solution set originally created by Mentor Graphics, especially the FPGA IC prototyping and testing platform, Veloce,[8] with vehicle scenarios and electronic systems configuration and validation testing provides an environment for accelerating performance engineering. Performance engineers use virtual electronics models to assess performance as electronic changes are made, rather than only as a final integration gate before release of the electronic devices. This allows the performance and electrical/electronic engineers to interact more often and earlier. Innovations happen faster when disciplines collaborate more.
PAVE 360 Benefits
PAVE 360 was introduced in mid-2019 and summarized in a CIMdata highlight entitled: PAVE 360 for Overcoming the Billion Driven-Miles Challenge of Autonomous Vehicles.[9] PAVE 360 assures the correct function of chips, electronics components, and support services, like operating systems, with just a subset of relevant tests cases. In the ideal world, this is decoupled from the application of electronics into a control system. But as seasoned automakers know, some new scenario (e.g., usage pattern, environment, or other unexpected phenomena) often uncovers a design weakness. Selection of relevant test cases helps determine criteria when considering design corrective actions. Determining only the tests needed is essential to stay competitive. PAVE 360 helps orchestrate newly discovered scenarios added to verification and validation criteria. By managing the scenarios need to cover adequate verification and validation testing, PAVE 360 helps automakers achieve product validation without billions of miles of physical testing—which was the practice with build and break test cycles in legacy automobile development.
Concluding Remarks
By providing verification and validation capabilities for cities and automakers, Siemens is helping bridge the ADAS to AV gap. CIMdata forecasts that automakers using Siemens’ AVD solution set will realize the needed growth and expansion of virtual, continuous performance engineering. This growth must not slow down product development, and with Siemens continuous verification and validation capabilities it will not. Rather the role will evolve and interaction of performance engineering with the other disciplines will be accelerated. Performance engineering remains a competitive advantage, and when accelerated correctly, product and process innovation are enhanced with continuous verification and validation. Performance engineering has always focused on what customers value—safety, fun, luxury, and lifestyles. The accelerating pace of changing consumer tastes, thus the need to adjust performance measures, will change at a faster and faster pace.
Siemens’ AVD solutions provide test creation and management capabilities for execution and correlation studies at all levels of AV design and operations, utilizing a combination of local and cloud-based technologies. Continuous verification and validation are cost effective when most of it is done virtually. When required, hardware subsystem and integration testing will be done with the same scenarios as those used in the virtual simulations. Siemens’ multi-domain information model has been designed to continuously provide an accurate digital twin of the vehicle design and its use. Siemens provides a solution set ready for AV development and operations—supporting AV upgrades with their SSE, Capital, and now AVD solutions.
CIMdata recommends that companies consider Siemens’ portfolio of solutions. With computers and their associated electronics being pervasive, a comprehensive, integrated yet heterogeneous, multi-disciplinary product development environment connected to all stakeholders (e.g., other vehicles and cities) is essential. Siemens’ ongoing advances in Quality Management,[10] Systems Engineering,[11] Simulation with Executable Digital Twins,[12] Connected Engineering,[13] Systems of Systems Frameworks,[14] and future concepts in the works are providing a rich set of capabilities that product designers will use for the next generation of mobility—autonomous vehicles. Finally, Siemens is addressing gaps in AV development with engineering tools that can be used to explore operational conditions as they are experienced. This is what happens when a leading solution provider builds on their strengths.
