VQGAN + CLIP + Unreal Engine + Surrealism
I enjoyed Rusty's explanation of the VQGAN + CLIP Unreal Engine trick in Today in Tabs yesterday (subscribers only, sorry). The gist is that people have found a way to improve images created with machine learning by including the phrase "unreal engine" (yes, the Mandalorian-powering video game technology) with their prompt when they ask a GAN to generate an image of that prompt.
Rusty included a link to a colab notebook that could run this particular GAN—VQGAN+CLIP—so I started playing around with it. I know next to nothing about AI or Machine Learning, but I can run code! I tried out the unreal engine trick and the results reminded me of Yves Tanguy paintings.
My next thought was, "why not feed it some lines from surrealist poetry"? (Or their English translations anyway.) So here are a few of those:
"a meaningless castle rolled along the surface of the earth."
– André Breton, Soluble Fish
"it is the star struck under my heel in the night."
– Robert Desnos, The Landscape
"The river I have under my tongue"
– Paul Eluard, The River
"where we live the flowers of the clocks catch fire"
– Tristan Tzara, The Great Lament of my Obscurity Three
The other day I linked to Nick Cave saying that AI lacks the nerve to create great music. How about visual art? Seeing machines struggle to make what humans make is interesting, but I don't think the results are inspired. Again, I have no idea how to tune these tools to get more interesting results and maybe further tuning or more attempts would create better images. Right now I think we haven't progressed beyond Hayao Miyazaki's thoughts on art by artificial intelligence. (spoiler alert: "I strongly feel that this is an insult to life itself.")
Rusty included a link to a colab notebook that could run this particular GAN—VQGAN+CLIP—so I started playing around with it. I know next to nothing about AI or Machine Learning, but I can run code! I tried out the unreal engine trick and the results reminded me of Yves Tanguy paintings.
My next thought was, "why not feed it some lines from surrealist poetry"? (Or their English translations anyway.) So here are a few of those:
"a meaningless castle rolled along the surface of the earth."– André Breton, Soluble Fish
"it is the star struck under my heel in the night."– Robert Desnos, The Landscape
"The river I have under my tongue"– Paul Eluard, The River
"where we live the flowers of the clocks catch fire"– Tristan Tzara, The Great Lament of my Obscurity Three
The other day I linked to Nick Cave saying that AI lacks the nerve to create great music. How about visual art? Seeing machines struggle to make what humans make is interesting, but I don't think the results are inspired. Again, I have no idea how to tune these tools to get more interesting results and maybe further tuning or more attempts would create better images. Right now I think we haven't progressed beyond Hayao Miyazaki's thoughts on art by artificial intelligence. (spoiler alert: "I strongly feel that this is an insult to life itself.")