Image synthesis with generative adversarial networks

Image to image translation is a class of image and graph problems where the goal is to learn the similarity between an input image and an output image using a training set of aligned image pairs. As a general-purpose solution to image-to-image translation problems, conditional adversarial networks have been investigated. These networks not only learn the mapping from the input image to the output image, but also learn a loss function to train this mapping. This makes it possible to apply the same general approach to problems that traditionally require very different loss formulations. This approach appears to be effective for colorizing images, synthesizing photos from label maps, and reconstructing objects from edge maps, among other tasks. Another generative model approach is used for image synthesis from text. Automatic synthesis of photo-realistic images from text could be useful and interesting, but current AI systems are still far from this goal. However, in recent years, general and powerful recurrent neural network architectures have been developed for learning distinctive text feature representations. Deep Convolutional Adversarial Generative Networks using this architecture demonstrate the model's capability by translating text characters into pixels to generate logical images. Finally, an estimated half of the world's languages have no written form, making it impossible for them to benefit from any existing text-based technology. We propose a speech to image framework that translates speech descriptions into photo-realistic images without any textual information, thus allowing unwritten languages to benefit from this technology.