Self-Driving Cars Get Better at Driving in the Dark

This article is part of our exclusive IEEE Journal Watch series in partnership with IEEE Xplore.

A team of researchers in China have uncovered a way to help autonomous cars “see” better in the dark, boosting the vehicles’ driving abilities by more than 10 percent. The secret to the researchers’ success lies in a decades-old theory on how the human eye works.

One way autonomous cars can navigate is to use a collection of cameras, which are each equipped with a special filter to discern the polarization of incoming light. Polarization refers to the direction of oscillation of light waves as they propagate, which can provide a lot of information about the object a light wave last bounced off, including the object’s surface features and details.

However, while polarization filters provide an autonomous vehicles with additional info about the objects surrounding them, the filter involves some pitfalls.

“While providing further information, this double filter design makes capturing photons at night more difficult,” says Jiandong Tian, a robotics professor at the Chinese Academy of Sciences in Shenyang. “The result is that in low-light conditions, the image quality of a polarization camera drops dramatically, with detail and sharpness being more severely affected.”

To overcome this problem, Tian and his student Yang Lu turned to a theory that attempts to explain why humans are able to discern colors relatively well under low-light conditions. The Retinex theory suggests that our visual system is able to discern light in two different ways—namely, the reflectance and illumination components of the light. Even in low-light conditions, our eyes and brain are able to compensate for changes in the illumination of the light enough to discern colors.

Tian’s team applied this concept to their autonomous-car-navigation system, which processes the reflective and luminescence qualities of polarized light separately. One algorithm—trained using real-world data of the same images in light and dark conditions—works like our own visual system to compensate for changes in brightness. A second algorithm processes the reflective properties of incoming light, removing background noise.

The researchers mounted cameras on cars to test their RPLENet model in the real world.Yang Lu

Whereas conventional autonomous vehicles tend to process only the reflective properties of light, this dual approach offers better results. “In the end, we get a [more] clearly rendered image,” Lu says.

In a study published 8 August in IEEE Transactions on Intelligent Vehicles, the researchers put their new approach, called RPLENet, to the test.

First, the team conducted simulations using real-world data of dim environments to verify that their approach could yield better low-light imaging. Then they mounted a camera that relies on RPLENet on a car and tested it in a real nighttime scenario. The results show that the new approach can improve driving accuracy by about 10 percent in experiments with autonomous driving algorithms.

Lu notes this new approach could lead to safer autonomous cars. “The excellent results we have achieved in our tests, especially in real nighttime scenarios, demonstrate the practical application potential of our method,” he says.

However, one challenge with the RPLENet approach is that it requires extensive training on datasets that are difficult to obtain (for example, images of the same object under different lighting scenarios).

“In the future, we plan to further explore weakly supervised and unsupervised learning methods to reduce the reliance on large amounts of labeled data,” says Lu. “This will accelerate the development of algorithms and help provide more efficient and cost-effective solutions in real-world applications.”

This story was updated on 9 September, 2024.

This story was updated on 10 September, 2024 to correct the location of the Chinese Academy of Sciences campus at which Jiandong Tian is based.