I have been experimenting with Google’s deepdream project in the last few days. Since I started posting images around (most prominently this collection of dreamscapes) people have been asking: how can I make my own? This short collection of notes represents an early attempt to gather up enough information for technically-minded people on Mac OS X to start dreaming.
Rather than compile from source I used Ryan Kennedy’s Docker container method. If you are new to Docker (like I was) you may find this guide to be helpful, assuming you already use Homebrew. From this we can assemble some rough steps to follow (but be sure to reference the original guides for more details):
- Install Homebrew.
- Install VirtualBox. You’ll need to download an installer and execute it the old-fashioned way. This is used to run boot2docker, a lightweight Linux distribution specifically designed for running Docker containers.
- Install Docker and boot2docker from the command line with:
brew install docker, and
brew install boot2docker.
- Initialize and load boot2docker with
boot2docker up. I also had to run some command line voodoo with
$(boot2docker shellinit)to get things working. If all goes well you’ll get an IP address you can use in the next step.
- Set a Docker environment variable with
export DOCKER_HOST=tcp://192.168.59.103:2375(or whatever IP address you saw in the last step).
- Fetch the deepdream container with
docker pull ryankennedyio/deepdream.
- Load the container with
docker run -d -p 443:8888 -e "PASSWORD=password" -v /[path]:/src ryankennedyio/deepdream. Be sure to replace
[path]with a valid path to a working folder. This will be accessible from within the Docker container (which means you can also store other models there).
boot2docker ipand navigate your browser to the IP address it returns (prefaced with
https://). Ignore the security warning, enter
password, and this should load the Python notebook originally released by Google.
After a system restart you will need to repeat most of the steps from 4 to 8 above (without
boot2docker init or step 6).
Actually using the notebook requires some trial and error if you’re not familiar with the conventions (as I wasn’t). Ryan Kennedy’s original post provides some basic tips on navigating the interface. In short, click on code blocks from the very top and hit the play button until you reach the code block that defines the
img variable. Here you will want to enter a filename that matches one in the path specified when the Docker container was originally loaded. If you want to skip right to the “inception” style image generation head down to the pair of code blocks that starts with
!mkdir frames and execute both in sequence. If everything is wired up correctly your computer will start dreaming.
How about customizing the output? I am no expert in Python or neural nets so I’m doing a lot of guesswork that might be foolish or wrong but I can relate a few things I’ve found.
First of all, to swap in another model you can download one from the model zoo, copy the files to your working folder, and alter the model definitions in the second code block like so:
model_path = '/src/models/googlenet_places205/' net_fn = model_path + 'deploy_places205.protxt' param_fn = model_path + 'googlelet_places205_train_iter_2400000.caffemodel'
These are the settings for Places205-GoogLeNet, another neural network used in the original post by Google. As the name would imply this one has been trained to recognize a dizzying variety of different places. In my admittedly limited experience thus far I’ve mainly seen regular features of manmade landscapes—temples, windmills, castles, and so on—as well as a variety of other scenes like baskets of fruit, piles of clothing, grocery store shelves, halls filled with chairs, and the like.
Whatever model you use you’ll want to execute
net.blobs.keys() to see what layers are available. These layers are then targeted by specifying the
end variable in the
deepdream functions defined under the “Producing dreams” heading. Here you may also wish to play with
octave_scale. Here I should note that I haven’t had any good results from setting both
end parameters to different layers. If you’re looking to dream coherent objects then you may wish to set them to the same layer.
How about the layers themselves, what do they produce? The emerging community of r/deepdream is a good place to look for stuff like this. Here is one test suite I found useful. In short, the layers of the default neural net optimize for slightly different dreams. In my experience I noted the following (and I will expand and correct these notes as I continue exploring):
inception_3a: mostly geometric patterns
inception_3b: still just patterns
inception_4a: eyes and some dogs
inception_4b: lots of dog faces
inception_4c: starting to get more into cars and buildings
inception_4d: more of a menagerie
inception_4e: lots of different animals; birds, snakes, monkeys, and so on
Of course, your results will be highly dependent on your source images (assuming you get outside of the “dog-slug” default layers). One run produced boats on an otherwise empty ocean stretching to the horizon which was quite cool.
Another note: the scale coefficient
s in the block of code that actually generates images can be adjusted to reduce the amount of cropping that occurs with every step of the feedback loop. The scaling is necessary to provide the neural net with slightly new information on each pass. If you’re using your own images you may want to allow for some trimming depending on how strong of an effect you are looking for.
If you would rather setup your own web service to drop images into try clouddream. I’m still working from the Python notebook for the time being it offers way more customization options.
Finally, if you’d like to check out some of my own work you can find a few early experiments on my main blog.