As stated by the authors of The AeroScapes Semantic Segmentation Dataset, they explore methods for learning across train and test distributions that dramatically differ in scene structure, viewpoints, and object statistics. Authors are motivated by the proliferation of aerial drone robotics, and consider the target task of semantic segmentation from aerial viewpoints. Inspired by the impact of Cityscapes, the authors introduce AeroScapes, a new dataset of 3269 images of aerial scenes (captured with a fleet of drones) annotated with dense semantic segmentation.

This dataset differs from existing segmentation datasets (that focus on ground-view or indoor scene domains) in terms of viewpoint, scene composition, and object scales. The authors propose a simple but effective approach for transferring knowledge from such diverse domains (for which considerable annotated training data exists) to the target task. To do so, authors train multiple models for aerial segmentation via progressive fine-tuning through each source domain. They then treat these collections of models as an ensemble that can be aggregated to significantly improve performance. Authors demonstrate large absolute improvements (8.12%) over widely-used standard baselines.

Traditional localization benchmarks focus primarily on object recognition in images, often neglecting the context in which these objects are situated. Background elements, such as terrain and aerial features, offer crucial semantic and geometric context to foreground objects. For instance, an autonomous car uses identified roads within its line of sight for navigation, avoiding parking attempts in sky or water areas. Hence, it’s vital to train terrain-based and aerial autonomous agents to recognize both foreground and background elements.

Real-time autonomous systems heavily rely on scene understanding to make decisions, requiring evaluation benchmarks to incorporate labeled image sequences. Agents using visual scene understanding must also integrate temporal information into their representations, making video data integration a necessity. The AeroScapes dataset comprises 3269 images from 141 video sequence, with some of it being temporally downsampled. The class distribution within AeroScapes reflects the common data imbalance in outdoor images, including both stuff and thing annotations, with the thing classes representing only approximately 1.51% of the data.

Aerial robots provide the advantage of exploring diverse environments and viewpoints that ground-based autonomous cars cannot access. This led to the creation of the AeroScapes Dataset, featuring images captured by drones at altitudes of 5-50 meters. These images come with segmentation maps, labeling both stuff classes (vegetation, roads, sky, construction) and thing classes (person, bikes, cars, drones, boats, obstacles, animals).