Gan Exploration

I have been taking a Machine Learning for Art class with Lia Coleman who I met through Seattle Creative Code a small group of artists who use code to make art. Most of this has been trained on Colab, but I’m currently setting up a machine learning rig so I’m not tied to the time limits of Colab.

WarpGAN

The header image is a group of photos that uses WarpGAN to generate caricatures of people. I started with my work badge to get a feel for it.

View this post on Instagram

WarpGAN #mlart #codeart #nightmarefuel

A post shared by Grayson Arts (@_graysonarts_) on

That’s pretty much nightmare fuel, so of course I really like it. Then I asked coworkers to submit photos, and Ben, one of my PMs, gave me a photo of him which results in this:

View this post on Instagram

WarpGAN explorations

A post shared by Grayson Arts (@_graysonarts_) on

Recently, my company laid off a bunch of good people even though the company had a record quarter, so of course, I decided to see what it would do with the CEO of the parent organization and the CEO of my company. This yielded some pretty cathartic renderings

View this post on Instagram

Art is catharsis

A post shared by Grayson Arts (@_graysonarts_) on

SinGAN

SinGAN allows you to take a single image and introduce movement and animation to it. I explored that a little with my badge photo, but didn’t save any of the output. It was interesting, but didn’t realize yield something productive. I might try again later.

Pix2PixHD Next Frame Prediction

Next Frame Prediction leads to some really interesting results. It takes a video and then trains the Pix2PixHD network on consecutive frames of the video. Pix2Pix is an unpaired image transfer. If you’ve seen the horse to zebra examples, that’s what it does, so the idea is if you can transfer the learning to consecutive frames of the video, you should be able to predict what the third frame will be.

I started off with the video from SinGAN and trained a model on that. Since that video was long, it took three days to train on Colab, having to restart every twelve hours. That was a pain in the ass which is what started me down the path of working on the training rig from old parts since I have a nice GPU that is unused.

After three days, this was the result:

View this post on Instagram

Next frame prediction is weird

A post shared by Grayson Arts (@_graysonarts_) on

Now, I’ll be the first to admit, most of my part involves sexuality in some way. I don’t view it as shock art or anything, but I think it’s an important subject to explore via art. So, with that being said, I filmed a short video of me naked. I have a love-hate relationship with my body, so much of the time I explore the edges of that relationship through art. Using this video, I trained a new model, but rather than feed it one of the source frames like was done in the video above, I decided to feed it an entirely different image.

That resulted in this which was converted to black and white to divorce it from the flesh tones that were in the original video.

View this post on Instagram

NFP with different seed photo from training data

A post shared by Grayson Arts (@_graysonarts_) on

I’m going to continue to explore this space. My friend Jesse asked me to make a music video for some of his songs that explore the concept of paramours, so I’m excited to explore that and feed pictures of past lovers into it as source material.