Build a driver monitoring system with Keypoint

Overview│ Building a Driver Assistance System with Keypoint Processing

Building a driver monitoring system utilizing facial detection technology is a project that has received a lot of attention recently for the purpose of preventing accidents on the road. Most often mentioned is the Driver Monitoring system, which uses cameras and machine learning algorithms to detect and analyze the driver's behavior, posture, and attention level, and can issue warnings or control the vehicle if necessary. This requires Keypoint labeling technology, which assigns pre-defined landmarks to images and objects in the video to draw meaningful shapes.

One of the key components of a driver monitoring system is Object Detection technology, which is the process of identifying and locating objects within an image or video. Commonly available in driver monitoring systems are implementations of deep learning-based facial detection technology.

Our client is a leading company in South Korea that leads the innovation of artificial intelligence algorithms. The company conducts research on computer vision and language understanding, as well as various machine learning algorithms, with the goal of raising the intelligence level of AI algorithms to human levels, and ultimately understanding users and environments.

In particular, AI algorithms can achieve a high level of understanding through visual data. It is difficult for a computer to understand an environment and its context through images. However, deep learning-based computer vision technology is enabling the development of applications from biometrics to augmented reality to driver assistance technology.

Problem│ Processing facial detection using Keypoint, the key is 'fast and accurate'

Object detection algorithms can be trained using large datasets. These datasets consist of labeled images and videos of drivers that can be collected in a variety of driving scenarios and lighting conditions. Using deep learning techniques, the algorithm learns to distinguish between safe and unsafe driving behaviors and recognize relevant features and patterns.

To implement a driver monitoring system, you can equip your vehicle with multiple cameras to capture video of the driver and their surroundings. The video data is then fed into a facial detection algorithm. After that, you need Keypoint, which analyzes the video frame by frame to detect, track, and visualize objects of interest, such as the driver's face, hands, or eyes.

In other words, the customer needed a Keypoint-labeled dataset for training the AI that would be implemented in the driver monitoring system.

In addition, their previous partner had been forced to use overseas labelers due to budgetary constraints, resulting in communication issues and poor dataset quality for the cost.

WithDataHunt, the client was expecting to improve the quality of the dataset at a competitive cost, and it was also important to complete the project as quickly as possible due to the delay caused by the change of partner.

Solution│ Building a DMS dataset with keypoint labeling

The client had two main requests. To accomplish this mission, the Datahunt project manager first developed a plan to solve the problem.

‍

• Plan 1. Hire a group of experienced data labellers
• Plan 2. Technical support to speed up the project

The main challenge was to kill two birds with one stone: quality and speed of work.To conclude, we actively leveraged the auto-labeling model to assist the labelers. We also selected a group of labelers who were familiar with the authoring tool (keypoint labeling) for the work task and had experience with similar projects.

Here's what DataHunt did

‍

• Guided the project through online training before committing to data processing.
• ~Selected operators with a pre-test to determine their skill level
• Keypoint/bounding box labeling of driver's facial areas
• Labeling of status values such as various actions taken by the driver/passenger while driving and whether or not they are wearing a belt
• Using the initial task data, apply a keypoint/bounding box model for automatic labeling of the passenger's face area.
• ~The model is able to assist the human operator with an accuracy of approximately 85%.
• ~The model was able to assist the operator with about 85% accuracy.

We minimized unnecessary time consumption by responding as quickly as possible to any issues that needed to be addressed along the way.

Issue1. Different workers had different standards for taking keypoints in the fine eyelid area of the face, causing work confusion.

Issue2. Found ambiguous or insufficient standards during the work process

->Prepare detailed standards in quick consultation with the customer and distribute them to workers to ensure data quality (e.g., specify the flesh that touches the whites of the eyes).

Issue3. Many images are dark and difficult to work with

->Provide brightness/sharpness adjustment function within the platform itself

Result│ Keypoint labeling and facial detection dataset, '99% accuracy'

‍

Belows are the project objectives that Datahunt originally promised to the client and the results that were achieved.

Improved data accuracy‍

• Realized 99% accuracy
• Passed all criteria without a single re-check

Reduced work hours

• Successfully reduced man-hours and overall project lead time
• Delivered in 9 weeks, 3 weeks shorter than the original desired delivery of 12weeks.

Communication

• Proactive communication between tasks ensured delivery without quality issues

How we achieved 99% dataset accuracy

Throughout the project, the client was highly satisfied with the high quality ofDataHunt's data, noting that "having a highly skilled team of data labelers as well as meticulous operations is one of DataHunt's greatest strengths."

We don't compromise on quality, and if we were to work with offshore workers with cheaper labor costs, we would have struggled with data quality even if all the conditions were met, but we were able to realize cost-effectiveness by using skilled domestic workers.

Although this was a project with a large company that is a domestic AI leader, I feel that we achieved a major milestone in that we were able to confidently achieve strict quality control standards and shorten the delivery time. We would like to showcase our 99% dataset accuracy to all clients who will be working with us in the future. Datahunt is always ready to help.