Nvidia Releases AI Painter Translating Words to Landscapes

Nvidia has unveiled a new AI tool that translates words into art. The GauGAN2 upgrades the GauGAN text-to-image model for transforming a written description into a realistic painting like the one seen above.

Word Painting

GauGAN2 takes any descriptive phrase and the AI will start drawing it. The real-time digital painting will shift based on additional adjectives and other descriptions, so the landscape can appear in different weather and or times, and the user can tweak it to match their vision. The tool can produce a segmentation map, a kind of outline, and then sketch or paint in details as it learns what the user wants to see in the photorealistic final version. The final version doesn’t need to match Earth at all either. Users can add strange sky colors, or even an extra sun, should they want to reproduce a landscape from Star Wars as Nvidia suggests. GauGAN2 can be tested on the NVIDIA AI Demos page right now.

“Rather than needing to draw out every element of an imagined scene, users can enter a brief phrase to quickly generate the key features and theme of an image, such as a snow-capped mountain range. This starting point can then be customized with sketches to make a specific mountain taller or add a couple trees in the foreground, or clouds in the sky,” Nvidia’s Isha Salian explained in a blog post. “Compared to state-of-the-art models specifically for text-to-image or segmentation map-to-image applications, the neural network behind GauGAN2 produces a greater variety and higher quality of images.”

Art AI

Nvidia’s tool is similar to what Yandex has already made for its Alice voice assistant. The artificial intelligence underlying Alice uses two neural networks to figure out what to draw based on what it is told in a voice command. Twentieth-century art and painting were used to train one of the neural networks, while the other analyzes and understands what is in the painting. The Yandex search engine collated responses to search queries comparing images and words as the training method for the neural networks. Building on those neural networks, Alice can also spot what is in a painting it is shown, and who created it.

Follow @voicebotai   Follow @erichschwartz