Contact Us
Once-for-All (OFA) Application in Edge AI Development for Automotive Manufacturers
1. project overview
AN AUTOMOTIVE MANUFACTURER WAS FACED WITH THE CHALLENGE OF OPTIMIZING AI MODELS FOR DEPLOYMENT TO THEIR OPERATIONAL SYSTEM, SINCE AI MODELS MUST BE DEVELOPED IN ADVANCE BASED ON FUNCTIONAL REQUIREMENTS, AND THEN OPTIMIZED TO MEET THE FUNCTIONAL REQUIREMENTS OF THE OPERATIONAL SYSTEM'S DEVICE SPECIFICATIONS. HOWEVER, IT WAS NOT EASY TO SECURE SKILLED PERSONNEL AND ASSIGN THEM TO THE PROJECT AT THE NECESSARY TIME FOR THE DEVELOPMENT OF SUCH AI MODEL OPTIMIZATION.
IN ADDITION, WHEN EDGE DEVICES WERE CHANGED DUE TO CHANGES IN REQUIREMENT SPECIFICATIONS AFTER THE START OF DEVELOPMENT, OR WHEN OPTIMIZATION TO SATISFY THE REQUIRED SPECIFICATIONS COULD NOT BE ACHIEVED, IT WAS NECESSARY TO GO BACK TO THE DEVELOPMENT PROCESS AND REDEVELOP THE AI MODEL. IN ORDER TO AVOID UNINTENDED INCREASE OF RESOURCES, COSTS, AND DEVELOPMENT TIME DUE TO DEVELOPMENT REWORK, AND TO OPTIMIZE THE AI MODEL TO MEET THE REQUIRED SPECIFICATIONS FROM A HUGE SEARCH SPACE, THE KNOWLEDGE OF SKILLED DEVELOPERS WHO ARE FAMILIAR WITH BOTH AI MODEL DEVELOPMENT AND EDGE DEVICE DEVELOPMENT WAS NECESSARY.
In response to these challenges, OFA OFA OFA can generate multiple models (subnets) with different architectures at once by learning a model (Supernet) that is a redundant representation of the AI model to be optimized once. The optimized model can be obtained by selecting the model with the appropriate performance for the target device. By extending this method to Araya, we have confirmed that it is possible to develop models with optimal performance for various edge devices with different characteristics in a highly efficient manner. We have also confirmed that AI model developers who are capable of learning and evaluating AI can efficiently optimize AI models even if they are not familiar with edge device development.
2. social background
CONVENTIONAL EDGE AI DEVELOPMENT REQUIRES A TEAM OF EXPERIENCED EDGE AI DEVELOPERS, AI MODEL DEVELOPERS, AND EMBEDDED DEVELOPERS, AND TAKES A REASONABLE AMOUNT OF TIME AND MAN-HOURS THROUGH A TRIAL-AND-ERROR PROCESS. HOWEVER, EXPERIENCED EDGE AI DEVELOPERS ARE SCARCE, AND IT TAKES A LONG TIME, COSTS A LOT OF MONEY, AND REQUIRES KNOW-HOW TO TRAIN SUCH PEOPLE IN-HOUSE.
IN THIS CASE, WE WERE CONSULTED BY A DEPARTMENT THAT IS ENGAGED IN THE ADVANCED DEVELOPMENT OF AI MODELS TO REALIZE NEW FUNCTIONS TO BE INCORPORATED IN FUTURE DEVELOPMENT MODELS OF VEHICLES. ALTHOUGH THE DEVELOPMENT OF ALGORITHMS AND AI FOR REALIZING NEW FUNCTIONS WAS BEING DONE IN-HOUSE, THE DEVELOPMENT OF EDGE DEVICES FOR OPTIMIZING THE DEVELOPED AI MODELS LACKED SKILLED PERSONNEL, IT WAS NOT EASY TO ACQUIRE HUMAN RESOURCES FROM OUTSIDE THE COMPANY, AND TRAINING AND RESKILLING OF IN-HOUSE PERSONNEL REQUIRED TIME AND MONEY. THE CHALLENGES WERE THESE.
TO ADDRESS THIS SENSE OF CHALLENGE, THE OFA-BASED METHODOLOGY DEVELOPED BY ARAYAAUTOMATES THE PROCESS OF AI MODEL OPTIMIZATION, ALLOWING AI DEVELOPERS WITH NO EXPERIENCE IN EDGE DEVICE DEVELOPMENT TO EFFICIENTLY DEVELOP OPTIMIZED MODELS FOR A VARIETY OF DEVICES.
3. features of development technology and how to proceed with development
This time, we conducted a proof-of-concept (PoC) of applying OFA-based methods to a system that uses image recognition AI to detect obstacles on the road from in-vehicle front camera images. The system consisted of multiple deep learning models with dependencies based on unsupervised learning, and had a problem with a large number of MACs in the models and long inference time.
THE APPLICATION OF THE OFA-BASED METHODOLOGY TO EACH MODEL WAS CONDUCTED IN TWO MAJOR PROCESSES.
- learning phase
- For the parameters of the target model (kernel size, depth, and channel width), a model (Supernet) with redundant representations of the most redundant parameters is prepared to train the entire model.
- The model is then trained step by step with parameters fine-tuned to the overall model depth, kernel size, and resolution.
- This allows for the efficient training of a large number (e.g., ~10^19) of models (Subnets) that flexibly accommodate a variety of parameters and input sizes.
- Deployment Phase
- Create an accuracy predictor and a speed predictor to evaluate the performance of a number of models obtained in the training phase.
- The accuracy predictor can be prepared at very low cost using the data set used in the training phase. The speed predictor simply uses the number of MACs calculated from the number of parameters in the model. It can also use estimated or measured execution speeds on the device to be deployed.
- Based on the accuracy and speed information obtained from the created accuracy and delay predictors, a model with the optimal performance (accuracy and speed) for the device to be deployed is obtained.
As we proceeded with the PoC, we sometimes encountered issues that were difficult to optimize by applying OFA-based methods to some models. Each time this happened, we proceeded with the PoC by repeating improvements at Araya. Specifically, we made changes to the model architecture using knowledge transfer through distillation, pre-training using open data sets, and also improved the sampling strategy used in the OFA process. Ultimately, we were able to obtain an optimized model that met the required specifications, confirming the effectiveness of the OFA-based approach.
Summary and Future Prospects
IN THE PAST, EDGE AI DEVELOPMENT REQUIRED DEVELOPING AI MODELS BUILT BY TRIAL AND ERROR FOR EDGE DEVICES, DEPLOYING THEM TO DEVICES TO CHECK THEIR ACCURACY AND SPEED, AND IF THE PERFORMANCE WAS NOT SATISFACTORY, THEN GOING BACK TO THE PROCESS OF AI MODEL PARAMETER SELECTION, MINIATURIZATION, ETC. TO DEVELOP THE MODELS AGAIN. BY USING OFA-BASED METHODS, A LARGE NUMBER (E.G., ~10^19) OF DIVERSE OPTIMIZATION MODELS CAN BE EXTRACTED IN A SINGLE MODEL TRAINING. OPTIMAL MODELS CAN THEN BE DEVELOPED AND DEPLOYED TO A VARIETY OF EDGE DEVICES WITHOUT THE RISK OF UNINTENDED DEVELOPMENT REWORK DUE TO UNFULFILLED SPECIFICATIONS. SINCE THIS DEVELOPMENT PROCESS IS A TECHNOLOGY THAT CAN BE EASILY ADOPTED BY AI MODEL DEVELOPERS, MORE DEVELOPERS WILL BE ABLE TO UTILIZE OFA-BASED METHODS TO DEVELOP EDGE AI. AS A RESULT, DEVELOPMENT EFFICIENCY CAN BE IMPROVED BY SOLVING THE ISSUES OF HUMAN RESOURCES AND INCREASED TIME AND COST DUE TO REWORK.
