Validation Team for an Automated Driving System.
Introduction
Due to the fact that ProGineer Technologies (www.progineer.net) has extensive experience in quality assurance, validation and Metric Driven Verification tools and methodologies, in 2019, ProGineer Technologies was approached by a leading high-tech company that is developing systems to Verify ADS (Automated Driving Systems) and autonomous vehicles, which is a significant challenging barrier to autonomous vehicle deployment.
The company was looking for a validation team to test and validate its product’s complex systems before sending them to their own clients. This team needed to work and function as an extension to their existing validation team.
The Story of the ADS Validation project
In addition to creating the most complete Verification & Validation (V&V) libraries in the market and the leading coverage-driven verification and validation platform for ADAS/AV development, ProGineer’s client is working to create millions of test scenarios for automated driving systems and autonomous vehicles. The client was looking for a technological partner that could provide experienced validation engineers with reasonable pricing and low turnover of resources.
ProGineer Technologies had gained experience in test coverage technologies and methodologies by working with one of its other multinational clients (Cadence Design Systems), which positioned ProGineer in a better place among its competitors. ProGineer then selected Lena, a senior female engineer, to lead the team after visiting the client’s offices and understanding their technical needs. Lena had seven years of experience working with ProGineer on another outsourcing project, so she had the necessary qualifications to lead the technical team. The client conducted an interview with Lena as part of ProGineer’s hiring process, and she then immediately began the engagement with her technical counterpart from the client’s team.
The first critical milestone was to learn the descriptive programming language written and developed by the client with the aim of creating abstract scenarios that describe the behavior of the AV as well as other actors in the environment. The language was recently adopted as the basis for ASAM’s OpenSCENARIO® 2.0.0 One of our major contributions at ProGineer was to take part in the validation of the hundreds of scenarios that form the V&V libraries. Visual Inspection for thousands of runs was necessary to assure that the intent of the scenarios was captured and the tool generated no bias of any kind. All data was then be uploaded to an analytic managerial web application. ProGineer’s team was involved from day one in testing the APIs and UIs, and building an entire automated regression suite that would keep this platform running at its best to conduct extensive reporting and analysis including safety and coverage metrics to guide completeness of AV testing. The team grew as the responsibilities increased. It all started with Lena in November 2019, and since then the team has grown up to 9 engineers doing various software Validation services on this complex system.
ProGineer’s validation team continued to deliver with high motivation and dedication even during the Covid-19 crisis. Since the work from home model was partially in place even before the epidemic, implementing it was immediate and smooth. All this was done with transparency and open communication with the client which made it easy for ProGineer and its clients to get through that tough period.