Neural Scene Decoration

Neural Scene Decoration from a Single Photograph

Hong-Wing Pang1           Yingshu Chen1           Phuoc-Hieu Le2           Binh-Son Hua2           Duc-Thanh Nguyen3           Sai-Kit Yeung1

1The Hong Kong University of Science and Technology 2VinAI Research 3Deakin University

European Conference on Computer Vision (ECCV), 2022

Given the empty scene along with bounding boxes or point labels, our model can generate high quality decorated scene. Generation results using box label format (top row) and point label format (bottom row).

Abstract

Furnishing and rendering indoor scenes has been a long-standing task for interior design, where artists create a conceptual design for the space, build a 3D model of the space, decorate, and then perform rendering. Although the task is important, it is tedious and requires tremendous effort. In this paper, we introduce a new problem of domain-specific indoor scene image synthesis, namely neural scene decoration. Given a photograph of an empty indoor space and a list of decorations with layout determined by user, we aim to synthesize a new image of the same space with desired furnishing and decorations. Neural scene decoration can be applied to create conceptual interior designs in a simple yet effective manner. Our attempt to this research problem is a novel scene generation architecture that transforms an empty scene and an object layout into a realistic furnished scene photograph. We demonstrate the performance of our proposed method by comparing it with conditional image synthesis baselines built upon prevailing image translation approaches both qualitatively and quantitatively. We conduct extensive experiments to further validate the plausibility and aesthetics of our generated scenes.

Video

Coming Soon!

Materials

Our Network Architecture

Overview of our generator (left) and discriminator (right). Convolution layers labeled with Dn2 halve the spatial dimensions of input feature maps using stride 2.


Qualitative Results

Generation results of our method and other baselines, using box label format (the first two rows) and point label format (the last two rows). Best view with zoom.


Quantitative Results
Citation
        @inproceedings{hwpang2022nsd,
  title={Neural Scene Decoration from a Single Photograph},
  author={Hong-Wing Pang, Yingshu Chen, Phuoc-Hieu Le, Binh-Son Hua, Duc Thanh Nguyen, Sai-Kit Yeung},
  booktitle={Proceedings of European Conference on Computer Vision (ECCV)},
  year={2022}
}

Acknowledgements

This paper was partially supported by an internal grant from HKUST (R9429) and the HKUST-WeBank Joint Lab. The website is modified from this template.