Joint Learning of Intrinsic Images and Semantic Segmentation

by   Anil S. Baslamisli, et al.

Semantic segmentation of outdoor scenes is problematic when there are variations in imaging conditions. It is known that albedo (reflectance) is invariant to all kinds of illumination effects. Thus, using reflectance images for semantic segmentation task can be favorable. Additionally, not only segmentation may benefit from reflectance, but also segmentation may be useful for reflectance computation. Therefore, in this paper, the tasks of semantic segmentation and intrinsic image decomposition are considered as a combined process by exploring their mutual relationship in a joint fashion. To that end, we propose a supervised end-to-end CNN architecture to jointly learn intrinsic image decomposition and semantic segmentation. We analyze the gains of addressing those two problems jointly. Moreover, new cascade CNN architectures for intrinsic-for-segmentation and segmentation-for-intrinsic are proposed as single tasks. Furthermore, a dataset of 35K synthetic images of natural environments is created with corresponding albedo and shading (intrinsics), as well as semantic labels (segmentation) assigned to each object/scene. The experiments show that joint learning of intrinsic image decomposition and semantic segmentation is beneficial for both tasks for natural scenes. Dataset and models are available at:


page 9

page 11

page 13

page 20

page 21

page 22

page 23

page 24


SemSegDepth: A Combined Model for Semantic Segmentation and Depth Completion

Holistic scene understanding is pivotal for the performance of autonomou...

EDEN: Multimodal Synthetic Dataset of Enclosed GarDEN Scenes

Multimodal large-scale datasets for outdoor scenes are mostly designed f...

ShadingNet: Image Intrinsics by Fine-Grained Shading Decomposition

In general, intrinsic image decomposition algorithms interpret shading a...

CongNaMul: A Dataset for Advanced Image Processing of Soybean Sprouts

We present 'CongNaMul', a comprehensive dataset designed for various tas...

Joint Facade Registration and Segmentation for Urban Localization

This paper presents an efficient approach for solving jointly facade reg...

A Bag of Words Approach for Semantic Segmentation of Monitored Scenes

This paper proposes a semantic segmentation method for outdoor scenes ca...

Resolving Language and Vision Ambiguities Together: Joint Segmentation & Prepositional Attachment Resolution in Captioned Scenes

We present an approach to simultaneously perform semantic segmentation a...

Please sign up or login with your details

Forgot password? Click here to reset