In this project I implemented the paper A Neural Algorithm of Artistic Style, Gatys et. al. 2015 using the Keras machine learning framework.
Cat photo credit: Claire Whittle
My implementation was loosely based on the fantastic Keras example code by Francois Chollet. In my implementation I modifed the VGG19 architecture using pre-trained weights trained on ImageNet. I replace the maximum pooling layers with average pooling using the same strides and discard the fully connected layers at the end of the network as they are not needed and take up unecessary memory on the GPU.
In Francois' code he makes use of the SciPy L-BFGS optimizer. While this produced nice results in a small number of iterations I found that the high memory requirement of L-BFGS (even though the L stands for Limited-memory) was prohibitive in producing images of a resolution higher than around 400x400. Through experimentation I found that the SciPy Conjugate Gradient optimizer provided good results with greatly reduced memory complexity, allowing me to raise the resolution of produced images to around 720p on a single NVidia 870m GPU.
I plan to revisit this project in the future implementing it entirely in Tensorflow. I may also investigate newer and more advanced methods that have been proposed since the publication of Gatys' seminal paper in this area.
Full code for this project is available here as a Gist.
In the remainder of this post I will show some of the images that I produced with the linked code.
Boat photo credit: John Whittle
Boat photo credit: John Whittle
Cat photo credit: Claire Whittle
Cat photo credit: Claire Whittle
Cat photo credit: Claire Whittle
River photo credit: Taken from the original paper
River photo credit: Taken from the original paper
River photo credit: Taken from the original paper.
River photo credit: Taken from the original paper