Data fusion is the process of integrating multiple data sources to obtain a more stable, accurate, and useful information than that provided by any single source. Autonomous vehicles feature many independent sensors which can provide the same information directly or indirectly. For example, a GPS and a lidar offer accurate localization. GPS does so directly, while lidar does so by applying its reading to a map of the area prepared in advance [go to localization]. A skillful combination of the two allows for a better understanding of the surrounding world.
A key part of the data fusion process is the determination of data acquisition timing. There are several approaches, from physical sensor synchronization, through the software control of data acquisition timing, to using a single time source in the system, which allows to clearly define temporal relationships between data.
Depending on the relationship between data sources, data fusion techniques can be classified as follows:
- Complementary – information from the sources describes different parts of the same scene, e.g., two lidars, one on the front and one on the back of the vehicle, help determine the vehicle’s omnidirectional environment.
2. Redundant – two or more sources provide the same type of information, e.g., two GPS sensors enable a more accurate localization.
3. Cooperative – two or more sources generate new information, e.g., two GPS sensors placed at the right distance from each other provide information on orientation.
Data fusion takes place at three levels of abstraction: measurements, characteristics, and decisions. The low level of measurements involves raw data fusion to obtain the same data with better parameters. The medium level of abstraction involves the fusion of characteristics and features (shapes, textures, their positions) to enable different features needed to perform other tasks such as object detection. The high level involves decision fusion, which uses symbolic representations, such as detected objects or trajectories, to work out a more probable decision (e.g., fusing object detection using different algorithms).
multi-level fusion