This post outlines the technical background of 2D to 3D conversion process. The text is from my undergraduate dissertation documentation. The sample conversion will be explained with screenshots in a different post shortly and will be linked to this post. I’ll add more links to some other tutorials as well.
2D to 3D conversion is done basically by horizontally offsetting the pixels of the monocular view with the use of a depth map to introduce horizontal disparity. Since there is no automatic process for depth map extraction from a video, depth maps are created by isolating each object in its related depth plane by rotoscoping and filling out the roto mattes with luminance levels relative to the depth level. Careful depth range and distribution preplanning is required to decide the gray levels of different depth planes.
Depth plane segmentation process is done with Mocha by creating individual track mattes of every object on the scene. Mocha estimates and track planar movements of shape contours regardless of the volume of the object. Therefore a single object is segmented on to multiple depth planes depending on their volume in the space. These track mattes could then be imported into Nuke or After Effects for depth map creation. To accurately fill information situated between depth planes, edges of adjacent roto mattes should be blurred into each other.
For objects which has a symmetrical depth distribution such as a soccer ball or a bottle, it is not necessary to create different depth layers. The tracked roto mattes could be filled with a linear reflected or radial gradient to precisely represent their volume. However minimum and maximum luminance levels of the gradient should not have a collision with luminance levels associated with other depth planes (Imagineer?Systems, 2011).
Nuke has a built-in 3D geometry displacement tool called ‘DisplaceGeo node’. This tool has the ability to displace a card mesh (3D mesh plane) with the use of a depth map. In 2D to 3D stereoscopic conversions this function is used to displace a card mesh towards the camera and project the original footage on to that mesh to re photograph the scene with a software 3D camera rig. The left camera is linked to the original footage while the right camera view is re photographed with the extruded geometry. To fill the missing parallax pixels, a technique called in-painting which efficiently smudge adjutant pixels to the gap is used (Seymour, 2011; Wright, 2010).
After Effects has a ‘Displacement Map’ effect which offsets pixels horizontally or vertically based upon the colour values of an 8-bit displacement map. The values ranging from 0 to 255 are converted into a -1 to +1 scale and the displacement amount is calculated by multiplying the converted value by the maximum displacement amount specified by the user. Colour values from 0 to 127 create negative displacement and values ranging from 129 to 255 create positive displacements. A value of 128 creates no displacement. This effect will be used in the treatment to develop and implement 2d to 3D conversion techniques (Adobe, 2010).
There are a few proprietary 2D to 3D conversion processes that used in the feature film industry. ‘Diamentialization’ patented by In-Three Inc. USA and View-D technology patented by Prime Focus, UK are two prominent 2D to 3D conversion processes. In both of these technologies, depth map creation is a process done with Mocha while proprietary methods are used to generate the right camera view.
References
Adobe (2010) Adobe After Effects CS5: Adobe reference. Available at: http://help.adobe.com/en_US/aftereffects/cs/using/index.html (Accessed: 28th March 2011).
Imagineer-Systems (2011) Learn Mocha: Videos. Available at: http://www.imagineersystems.com/videos (Accessed: 28th March 2011).
Seymour, M. (2011) Art of Stereo conversion: 2D to 3D. Available at: http://www.fxguide.com/featured/art-of-stereo-conversion-2d-to-3d/ (Accessed: 27th March 2011).
Wright, S. (2010) ‘Learn mocha: Chapter 8. 3D conversion workflow’ Imagineer systems [Video]. Available at: http://www.imagineersystems.com/videos/learn-mocha-3d-conversion/view (Accessed: 28th March 2011).
- Kaveen
