Citation

Tan, Yong Xuan and Lee, Chin Poo and Neo, Mai and Lim, Kian Ming and Lim, Jit Yan (2023) Text-to-image synthesis with self-supervised bi-stage generative adversarial network. Pattern Recognition Letters, 169. pp. 43-49. ISSN 0167-8655

Text 1-s2.0-S0167865523000880-main.pdf - Published Version Restricted to Repository staff only Download (2MB)

Abstract

Text-to-image synthesis is challenging as generating images that are visually realistic and semantically consistent with the given text description involves multi-modal learning with text and image. To address the challenges, this paper presents a text-to-image synthesis model that utilizes self-supervision and bi-stage image distribution architecture, referred to as the Self-Supervised Bi-Stage Generative Adversarial Network (SSBi-GAN). The self-supervision diversifies the learned representation thus improving the quality of the synthesized images. Besides that, the bi-stage architecture with Residual network enables the generation of larger images with finer visual contents. Not only that, some enhancements including L1 distance, one-sided smoothing and feature matching are incorporated to enhance the visual realism and semantic consistency of the images as well as the training stability of the model. The empirical results on Oxford-102 and CUB datasets corroborate the ability of the proposed SSBi-GAN in generating visually realistic and semantically consistent images.

Item Type:	Article
Uncontrolled Keywords:	Text-to-image-synthesis, Generative adversarial network, Self-supervised learning, GAN
Subjects:	Q Science > QD Chemistry > QD146-197 Inorganic chemistry
Divisions:	Faculty of Creative Multimedia (FCM) Faculty of Information Science and Technology (FIST)
Depositing User:	Ms Nurul Iqtiani Ahmad
Date Deposited:	02 May 2023 08:10
Last Modified:	31 Oct 2023 03:25
URII:	http://shdl.mmu.edu.my/id/eprint/11397

Downloads

Edit (login required)