How Pixel Differences can Affect Sensors in Self-driving Cars

Figure 1
Figure 1

About the Project

The FLIR team is working on real time object detection models with a primary use for autonomous driving cars. Much of the current technology is centered around RGB cameras as seen in Figure 1. 

While RGB sensing is a de facto standard in real time object detection models it is not without its weaknesses and must be used in conjunction with other sensing technology to provide reliably safe object detection for autonomous driving.

Figure 2
Figure 2

Figure 2 depicts a scene behind the lens of an RGB sensor on the left and the object detection of a thermal sensor on the right. RGB sensing tends to fail at night due to the reduced color space, which poses a large risk for the safety of autonomous driving cars. FLIR as a company specializes in the design and production of thermal imaging cameras, components, and imaging sensors, and hopes to leverage that technology in the field of computer vision.

This project is primarily concerned with analyzing the robustness of a thermal object detector for ADAS (Advanced Driver Assistance Systems) subject to the effect of added noise. Understanding the effect of noise is incredibly important to the field of computer vision. It has been observed that small amounts of noise can result in severe misclassifications. The team is also exploring adversarial attacks (adding inputs to a model to intentionally cause misclassifications) in an attempt to make the object detection models as robust as possible.

  Student Team


  • Louis Tremblay, Sponsor
  • Joshua Bang, TA
  • Sang-Yun Oh, Faculty

About FLIR

FLIR Systems, Inc. designs, develops, manufactures, markets, and distributes technologies that enhance perception and awareness. We bring innovative sensing solutions into daily life through our thermal imaging, visible-light imaging, video analytics, measurement and diagnostic, and advanced threat detection systems.

FLIR offers a diversified portfolio that serves a number of applications in government & defense, industrial, and commercial markets. Our products help first responders and military personnel protect and save lives, promote efficiency within the trades, and innovate consumer-facing technologies. FLIR strives to strengthen public safety and well-being, increase energy and time efficiency, and contribute to healthy and intelligent communities.