Semantically Multi-modal Image Synthesis

Zhen Zhu*    Zhiliang Xu*    Ansheng You    Xiang Bai   

Huazhong University of Science and Technology    Peking University   

in CVPR 2020

Arxiv    PyTorch


In this paper, we focus on semantically multi-modal image synthesis (SMIS) task, namely, generating multi-modal images at the semantic level. Previous work seeks to use multiple class-specific generators, constraining its usage in datasets with a small number of classes. We instead propose a novel Group Decreasing Network (GroupDNet) that leverages group convolutions in the generator and progressively decreases the group numbers of the convolutions in the decoder. Consequently, GroupDNet is armed with much more controllability on translating semantic labels to natural images and has plausible high-quality yields for datasets with many classes. Experiments on several challenging datasets demonstrate the superiority of GroupDNet on performing the SMIS task. We also show that GroupDNet is capable of performing a wide range of interesting synthesis applications.

Video of Semantically Multi-modal Image Synthesis

Related Work

Thanks to other Demonstrations

  • Can We Make An Image Synthesis AI Controllable?
  • CVPR 2020 | 妙笔生花新境界,语义级别多模态图像生成