Cross-view Semantic Segmentation for Sensing Surroundings

Bowen Pan1,2, Jiankai Sun2,3, Alex Andonian1, Aude Oliva1, and Bolei Zhou3
1Massachusetts Institute of Technology, 2Shanghai Jiao Tong University
3The Chinese University of Hong Kong


We introduce a novel spatial understanding task calls Cross-view Semantic Segmentation. The objective of cross-view semantic segmentation is to segment the top-down-view semantic masks from the first-view observations. We experiment our model without training on real data in two common scenarios: (a) Indoor-room scene. (b) Driving-traffic scene.

GIF Highlight (Driving-traffic Scene)

Demo Video