We are announcing the release of Applied Intuition’s Neural Sim, an AI-powered simulator that provides scalable, realistic, and controllable closed-loop simulation for next-generation advanced driver-assistance systems (ADAS) and automated driving (AD). With Neural Sim, teams can accelerate development by rapidly transforming drive logs into virtual scenarios using automated AI pipelines.
Challenges
Complexity is the key technical challenge for next-generation ADAS and AD. These systems require intricate features such as automated lane change and traffic jam assist. They deploy to diverse operational design domains (ODDs), including highway and urban environments. Additionally, the issue of liability can be complex, with the original equipment manufacturer (OEM)—not the driver—assuming responsibility in certain conditions. To handle these compounding complexities, OEMs rely increasingly on machine learning (ML) methods for perception, behaviors, and end-to-end systems. However, these next-generation systems require next-generation tools.
Existing log playback and re-simulation approaches scale well and offer perfect realism for open-loop evaluation. However, they do not support closed-loop sensor-level simulation because any significant pose divergence—the difference between simulated pose and logged pose— invalidates the sensor data.
Existing synthetic simulation methods, on the other hand, support closed-loop simulation, with fine-grained controls and the ability to create environments that might not have been encountered frequently in the real world. However, procedural generation of detailed 3D environments requires manual effort and time.
Both log re-simulation and synthetic simulation are used to develop production ADAS and AD systems today, but there is a need for a more scalable closed-loop simulator designed for end-to-end systems.
Applied Intuition’s Solution
Neural Sim is an AI-powered simulator that combines the scale and realism of log data with the controllability and efficiency of virtual testing, giving autonomy engineers multiple benefits.
Develop end-to-end ADAS with closed-loop simulation at scale
Neural Sim allows autonomy engineers to run simulations at the sensor-level, meaning engineers can generate data from camera, lidar, and other sensors. Moreover, it enables closed-loop simulation where the system under test can modify the ego pose, and the simulator will render valid sensor data. Finally, Neural Sim does this at scale by generating virtual worlds from drive logs. The result is the large-scale and high-throughput closed-loop simulation required for end-to-end ADAS.
Accelerate development with automated AI pipelines
Neural Sim also enables development teams to operate rapidly and efficiently. Neural Sim uses automated pipelines to rapidly transform drive logs into neural reconstructions in hours, not weeks. Moreover, because of this automation, development is efficient and does not bottleneck on teams of technical artists who are creating environments and worlds manually. Instead, teams can integrate neural simulation scenario extraction as an automated process during drive ingestion and triage. The result is faster development with simulation that scales as a function of drive data collected.
Reduce on-road testing with realistic sensor simulation
Neural Sim is critical not only for development, but also for testing and validation. Current methods often require on-road testing for end-to-end systems and integration of modules like perception and planning, an approach that can delay detection of issues. Neural Sim enables validation engineers to “shift left” by moving testing from on-road scenarios to neural simulations. The result is earlier testing, higher quality ADAS, and, ultimately, safer vehicles.
