Monday, 25 January 2016

Louis & Ethan - Ideas

Last time, I posted the result of an experimental technique for converting the following image to 3D. In this post I want to explain how it was done, and then finish by talking about my ideas for the animation.

The main selling point of the effect is that it perfectly transitions from the original picture to the 3D environment. There are two main things going on here. I guess you would call it Perspective Projection Mapping, combined with Displacement Mapping. Normally you model 3D objects based on front and side view images, (orthographic views), however what I have done is projected the image from a perspective camera.

Firstly, I cut the image into separate layers in Photoshop. The hidden (overlapping) areas needed to be filled in. For this test I simply 'content aware filled' the gaps.

The four layers

Then I created displacement maps for each layer. (Although I ignored the background layer in this step.) This can be achieved by converting the image to black and white, and then making many copies of the image at different blur levels, and mixing them together. It's not very scientific, but it works quite well.

Colour vs Displacement

Then it's a matter of positioning the layers in 3D and displacing them. It is important that both the colour image AND the displacement map are projected through the camera. This ensures that everything lines up straight away.

Layers arranged in 3D and displaced

In order for the effect to work well, the displacement maps need to accurately represent the depth in the original image. This is something I am still working on to improve. I know that I may need to re-sculpt areas manually to further improve the look.

Lastly, I used a second copy of the camera to fly through the 3D image and I cross-faded the shaders from using the solid colour of the original image alone, to a fully shaded effect. The result is a seamless transition from a 2D image, to a fully 3D scene.

Dreams are the central theme of the picture. The painted head is the dreamer. Based on Louis's feedback, I need to look into creating dream like effects and morphing. Maybe even disorientating effects (this may also influence the sound-scape I create for the animation). Really, anything can happen in a dream so the possibilities are endless.

The picture doesn't tell a story as such, so I wont be trying to make a story out of it, but I do see this as an opportunity to get creative and let people just enjoy the visuals. I imagine the animation will be divided into 3 phases.

In the first phase, I want to explore the picture in 3D as it currently stands, finishing with the dreamer at the bottom of the picture. (15 to 20 seconds)
Then in the second phase, I want to disrupt things, bring things to life and just mess with the picture in whatever ways I can think of. It's a dream after all! (45 to 55 seconds)
Then in the final phase, I want to transform everything back to the way it was in the beginning. A sort of reversing or undoing effect. (15 to 20 seconds)

I have already decided I want to play with the lighting. Maybe I can try changing the time of day?

I have also thought about taking a smooth (photoshoped) version of the following sculpture and deforming it into it's current, messy, state.

I could also try morphing some of the other sculptures. Maybe changing their expressions or swapping faces with one another.

I have also wondered about those random orange drops of colour.

Maybe these solid drops of colour could splash onto other sculptures and inject them with that colour. Maybe the 'balloon head' thing (not sure what they are, but I'm going to call them balloons) could drop onto the sculptures bellow which then undergo a transformation of colour.

I do have other ideas, but it's not so easy to picture them in words so the next step will be to try some of them out, and then assemble some kind of animatic from them. Keeping the creative process in 3D should make it easer for readers to understand my ideas, and for me to get a feel for their complexity to execute.

Ethan Shilling


  1. Fascinating stuff from you Ethan - as always! I know that Alan would be very interested in what you're up to here - and likewise our students who are often interested in turning concept art into '3D' environments etc. Looking forward very much to the next update!

  2. This is absolutely marvellous, Ethan! This work finally came alive for me. We looked at it - a 2D representation - a number of times, when we in the MASK Prize received it and then again when the Judges considered it for the prize. Never did we see it like this! Suspecting the work was powerful, we could not see it in such focus and detail. Thank you for unravelling its ideas for us! This is the power of animation, and your own superb understanding and artistry! Alla Tkachuk, The MASK Prize