Scene and Motion Detection to Eliminate Discomfort
A simple and effective implementation of the Pulfrich 2D to 3D conversion algorithm takes two time delayed video frames and maps them to the left and right eye views.
However, such an implementation requires that there is good correlation between the video frames selected for the left and right eye views, otherwise the 3D effect stops working for those frames. Moreover, large scale frame to frame changes, that translate to large discrepancies in the left and right eye views, also cause a feeling of discomfort or disorientation for the viewer. This typically happens when the source video contains fast moving action, generating excessive change from video frame to video frame, or where there is a wholesale change of scene.
The algorithmic enhancement discussed below aims to eliminate any sense of discomfort or disorientation due to rapid motion or scene changes. It does this by:
- Detecting rapid motion and scene changes between frames.
- Ensuring that, for those frames impacted, the same view for the left eye and right eye is retained.
Retention of the same frame for the left and right eye, during rapid motion or scene changes, completely eliminates the risk of disorientation as the view temporarily reverts to a stretched 2D view for that period. The video samples below provide a demonstration of the algorithmic enhancement in action.
A further enhancement, involving tuning and motion compensation of the right eye frame compared to the left is described here.
The first video shows a 2D to 3D converted version of the "Wakanda Forever Trailer". It uses the Pulfrich base algorithm, but in addition a mask has been generated where rapid motion or scene changes are detected in the video. The mask shows up as a pink frame allocated to the right eye (in bottom half of the video). The format of this video and the videos below is 3D Half Top-Bottom (HTB). You can view this page in a VR Headset with a browser (such as the Meta Quest browser) to watch them in 3D. Make sure that you select the 3D Top/Bottom option to view them.
For more full resolution examples for download, see here:
Half TB Format (MKV) with Mask
The next video shows the result of replacing the pink mask, generated for the right eye, with the same frame content as used for the left eye - effectively returning the video to 2D for those frames:
Half TB Format (MKV) with Mask Replaced
The next video shows the result using the original base Pulfrich algorithm with no motion or scene detection for comparison purposes:
Half TB Format (MKV) No Scene Detection
I have also assembled a collection of low resolution converted trailers as further examples here.
The 2D source trailers can be found at the links below:
