Searching for Contextual Subtasks for Semantic Segmentation

McLaughlin, Connor

Etd

Searching for Contextual Subtasks for Semantic Segmentation

Public

Deep learning approaches for semantic segmentation have achieved tremendous success through their ability to model long-range scene context. However, by training for per-pixel classification, these methods fail to address the issue of class imbalance in segmentation datasets. Our work approaches segmentation from a contextual modeling standpoint by introducing novel subtasks as a human-provided hint or an auxiliary training signal. We first validate candidate subtasks through human-in-the-loop techniques to correct mistakes in segmentation, improving the mIoU of UPerNet-ResNet50 from 42.05 to 48.70 without any trained parameters. We then demonstrate the potential for multi-task learning of these subtasks with segmentation through a study of task gradients and end-to-end training.

Creator