2D to 3D with Universal Media Server (UMS) - Download Now
I have modified the Open Source Digital Media Server, UMS, so that you can browse your 2D video collection, select a video and watch it in 3D. The video is converted to 3D, as you watch it, in real-time by your PC/Server.
The UMS solution serves the converted video to 3D/VR Media Players via DLNA. Supported DLNA Media Player clients, that run standalone on VR Headsets, include; BigScreen or Pigasus. For PCVR use, VLC is recommended, as it also supports DLNA.
DLNA imposes certain restrictions. It requires the video to be encoded for transmission, which means that the PC hosting UMS needs to be powerful enough to handle both the 2D to 3D conversion task as well as encoding, which is particularly challenging for very high resolution videos (e.g. 4K). DLNA also does not support fast forward and rewind well for transcoded content such as generated by the 2D to 3D conversion process. To overcome this latter item, UMS can be configured to generate periodic chapter links as described in the guide below. For an optimised PCVR based 2D to 3D video conversion solution, that does not employ DLNA, and can better handle very high resolution videos, e.g. 4K, consider the Stream to 3D solution as an alternative. Stream to 3D can also save your 3D converted video as well as perform frame rate conversion.
This enhancement is the focus of Release 13.x of UMS. You can download it for free now from the UMS release site below. Release 13.2.1 is currently available for free public download:
This page provides installation instructions and usage guidance, focused on the 2D to 3D aspects of UMS. For broader information on UMS, consult the website.
Demo
The main use case considered is viewing 2D Videos in 3D within VR Headsets such as the Meta Quest 2 via media players such as the BigScreen VR Player , Pigasus VR Player or Virtual Desk Top (plus VLC) with on-the-fly server-side conversion to 3D.
The video below provides a brief demonstration of the solution using BigScreen VR standalone player (not the Stream VR version). The example input video is a 2D version of the "Avatar: The Way of Water" trailer:
Example of 3D Video Output:
The demonstration video above does not provide a complete view of the quality of the 3D output. For that, you can download a full resolution version of the example video in different formats from here. It is a converted example of the 2D version of the "Avatar: The Way of Water" Trailer, after applying the 2D to 3D conversion algorithm. The conversion uses the latest version of the same algorithm as the on-the-fly real-time conversion within UMS.
You can download it from the browser in your VR headset and then play the video locally within your VR media viewer of choice. This will give you the best view of the quality of the 2D to 3D conversion that you can expect.
Hardware and Software Requirements:
Hardware:
- A VR Headset (an Oculus/Meta Quest 2 VR Headset has been used for proving)
- A capable Windows PC with high-end NVIDIA or AMD GPU and CPU (an MSI GF65 Thin Laptop has been used for proving). This particular laptop has an NVIDIA GeForce RTX 3060 GPU.
- A high-speed home Wireless Network (the AX1800 Whole Home Mesh Wi-Fi 6 System has been used for proving)
Software:
- BigScreen VR Player , Pigasus VR Media Player or Virtual Desktop with VLC Media Player: N.B. the solution has been well proven with the standalone headset version of BigScreen (in a Quest 2 headset). The Steam VR version of BigScreen has also been reported as working, but seems to require signicant machine resources to run with UMS on the same machine, some users have reported issues with this configuration. For PCVR use Stream to 3D is a better solution.
- Specific UMS renderer configuration files have been provided for the BigScreen and Pigasus media players, support for additional players, known as renderers, can be easily added with additional appropriate configuration files.
- Virtual Desktop is supported in combination with VLC Media Player (note that an additional setting is required in the UMS VLC renderer configuration file to enable 3D conversion, described later in this article).
- The key requirement is for a media player that supports 3D playback and DLNA/UPNP. A range of headsets from Meta, Valve and Steam VR support such media players.
- Universal Media Server : A free DLNA, UPnP and HTTP/S Media Server. Release 13 or greater includes the 2D to 3D functionality.
- AviSynth+ (64 bit): A “Frameserver” video processing tool with scripting, a modern fork of Avisynth with multithreading and 64-bit support. It is recommended to install the 64 bit version that includes the vc redistributables.
- K-Lite Codec Pack: A collection of DirectShow filters, VFW/ACM codecs, and tools. Codecs and DirectShow filters are needed for decoding audio and video formats (with GPU hardware acceleration). The standard version of the download should be sufficient for most users.
Installation
You will receive some standard warnings from Windows when installing the required PC software from the internet; AviSynth+, the K-Lite Codec Pack and UMS. You should review these warnings and make sure that you are content to continue downloading the software. To pass the "Windows protected your PC" warning click "More info" (not so obvious) and "run anyway" if you are content to do so. You will also get the usual "Do you want to allow this app to make changes to your PC?" warning, accept and continue if you are comfortable to do so. This is Microsoft being careful on your behalf.
AviSynth+:
Begin by installing AviSynth+ from the link below:
Use this version rather than the AviSynth 32 bit version that currently ships with UMS as it has higher performance and is being actively maintained. UMS will detect this pre-installed version and use it rather than the one that it is shipped with.
Click through the setup wizard, keeping the defaults in terms of components to install. Click finish when installation completes.
K-Lite Codec Pack:
Install the standard version of the K-Lite Codec Pack from here:
Click "Download Standard" on the first page and choose a server to download from on the next page:
Run the setup and click through the installation screens. There are a lot of setup options that you can modify, or not, depending on your specific requirements. The install wizard will detect your specific GPU and machine setup. It does a pretty good job of auto-detecting what is needed for your specific PC, so it is recommended to go with the defaults.
Universal Media Server (UMS) with 2D to 3D Mod:
Install UMS from the official link below. UMS Release 13 or higher is required:
Click through the setup screens shown below:
The maximum memory size must be set to at least 4096 megabytes during install, as shown, or the resume video function will not work as effectively for your 2D to 3D converted videos (this is the default in recent versions of UMS):
Choose English as the language as translations for the 2D to 3D descriptions for other languages have not been completed:
Click Yes, to run the first-time configuration wizard:
Important: Click No, to "Should UMS hide its advanced options?":
It is recommended to click Yes to the next option, with respect to scanning folders on startup, but this may need some experimentation. If you have a lot of videos you might want to click No, as this could slow things down when first starting UMS at the start of a video viewing session, depending on the speed of your machine and whether the videos are local to your machine or on your home network:
Choose a folder containing some 2D video content to get started:
This version of UMS defaults to using the Web UI for configuration settings (there is also a thick client). UMS will start by displaying a browser Window and ask you to create an Admin user:
UMS will then detect "renderers" on your network. This set of devices (or media player software) could include compatible TVs that can display DLNA/UPNP video content sourced from UMS or media players running in your headset.
If UMS detects a component that could, in theory, display transcoded video, but for which it does not have an explicit configuration, it will show it as an "Unknown renderer".
In the case below, UMS has detected an "Unknown renderer" on my home network and Big Screen VR Player (which happened to be running in my headset at the time of UMS installation and start-up):
The default configuration settings for UMS need to be changed to enable the 2D to 3D conversion functionality. This is done in the "AviSynth/FFMpeg" transcoder settings page. To do this, click on the three line menu in the top right corner of the Web UI and select "Server Settings" as below:
Next, click on "Transcoding Settings":
Then navigate to "Video File Engines" under "Transcoding Settings" and select "AviSynth/FFmpeg" in the left hand side menu. This is where the main 2D to 3D Conversion settings are maintained:
Important: You must enable"2D to 3D video conversion" mode by clicking on the check box about half way down the page as shown in the enlarged view below. Remember to click "Save" also:
As you can see, this page also controls the conversion algorithm, frame stretch factor (related to IPD and extent of 3D effect), a lighting depth setting related to the second conversion algorithm, the 3D Output Format (note the default is Half Top-Bottom - which is not the highest resolution, but a good starting point to test from in terms of performance and compatibility, for the highest resolution use full SBS or TB) and finally a setting that allows you to scale the video down before it is converted to 3D (if for example you are experiencing performance issues). We will leave the settings at their default values for now.
It is recommended to also disable the "Video File Engines" other than "AviSynth/FFmpeg" to avoid cluttering up the UIs of media players (unless you want to use them of course). This is done by clicking on the green triangular play arrow for each engine so that it changes to a red disabled icon.
Next, under the "Video File Engines" menu, go the "FFmpeg Video" section. This is where you can set the video encoding acceleration method for AVC/H.264 video, which is the setting that will be used to encode the 3D video before sending it to 3D media players such as BigScreen VR Player etc.
The BigScreen and Pigasus renderer configuration files specify MPEGTS-H264-AC3 as the required format. Changing the H.265 settings will have no impact therefore on these players.
The default setting is "libx264", which means a software based encoder that provides excellent quality encoding and small file sizes, but will only use your PC's CPU (not GPU) for encoding:
Depending on the speed of your overall system, the "libx264" setting may well be the best setting to use in terms of quality, but if you do encounter performance issues (e.g. buffering) and you have an NVIDIA or AMD GPU, then it is recommended to switch to hardware based GPU specific encoding. This is faster, but can generate larger file sizes and in some cases may not produce as high a quality output as the "libx264" option, so it best to start with "libx264":
For hardware GPU video encoding acceleration with modern NVIDIA graphics cards use the "h264_nvenc" setting. For modern AMD cards use "h264_amf". The screen shot below shows "h264_nvenc" selected for NVIDIA. The AMD setting is less well proven (feedback appreciated):
Next, under the "Common Transcode Settings" section, make sure to enable "Chapters support" as highlighted in the screen shot below. This is important as most DLNA media viewers do not support fast forward and rewind functionality for Transcoded videos. Enabling chapters support will allow us to instead step through the generated video via a set of chapter links spaced at intervals through the video. This interval defaults to every 5 minutes, but you can change it to whatever interval works best for your video content, see screen shot:
Another setting that is worth highlighting is in the "General Settings" tab, where you can set an IP filter as shown. That is, you can specify a list of IP addresses for which UMS will detect renderers and ignore any other devices on your network. I use this to specify the IP address of my VR headset so that only media players on my headset are detected. I have also configured my router so that my VR headset is given a fixed IP address.
Important: There is a bug in UMS Release 13.0.0 where it may not immediately activate all settings changes in the Web UI until the UMS server is restarted. This is fixed in later releases. To ensure settings are picked up correctly, you can trigger a server restart through the "Tools" menu as shown in the screen shot below, just select "Restart server", the restart should only take a few seconds:
Finally, you may encounter some difficulty with UMS in not correctly identifying your media player of choice. For example, I have recently noticed that the latest update to Sky Box VR Player has changed its User Agent signature. More significantly, with this player update, it now seems to behave inconsistently in terms of media folder/path listings with respect to UMS (particularly with respect to the virtual "#--Transcode--#" folder). It is therefore probably best to avoid Sky Box VR Player for use with UMS right now.
BigScreen or Pigasus is recommended instead. If necessary, you can force UMS to use a specific renderer configuration via the screen below, taking Sky Box VR Player as an example:
That completes the main installation and configuration of UMS.
In terms of the best way to view the 2D to 3D converted videos, there are options. If you have a choice, I would recommend the Virtual Desk Top (with VLC) option as being the most likely to deliver the highest quality output, with the fullest set of functionality for most users, the rationale for this is detailed in the next section.
If the Virtual Desktop solution is not an option for your system setup then you can also use your VR headset very effectively with one of the supported media players.
At present BigScreen and Pigasus are the best options. These players will detect UMS as a DLNA media server when the appropriate DLNA media server selection is made in the relevant media player menu. You will then be able to browse to your selected video via the "AviSynth/FFMpeg" Transcoder selection within the "#--Transcode--#" virtual folder presented to the media player by UMS. You will also be able to quickly step through your videos to different points in time using the generated chapter links in the transcode folder.
Pigasus and BigScreen both have different strengths:
- Pigasus supports a wide range of 3D output formats including Full SBS and Full TB/OU formats for the highest resolution ouput, it lets you resize the screen, supports chapter links and you can also fast forward at up to 3 times speed (or indeed run in slow motion). It also supports the native aspect ratios of videos without transforming them.
- BigScreen supports chapters and resume functionality well, it has more limited support for 3D formats than Pigasus, but supports Half SBS and Half Top/Bottom formats well. It has a range of wonderful environments in which to watch your videos. These environments do not let you change the size of the screen as the screen size is fixed in each environment, however the size of the screen does vary between environments - so you can pick an environment with a screen size that you like. All of the environments force an aspect ratio of 16:9 (Widescreen format) on the videos that you watch (like your Widescreen TV). UMS will automatically adjust the aspect ratio of videos to match this requirement.
- BigScreen is free whereas Pigasus is a paid product.
UMS currently has renderer configurations to support both BigScreen VR Player and Pigasus VR Media Player. The articles below illustrate how to use UMS from them:
Recommended Approach: Virtual Desktop with VLC:
As mentioned above, if you have a choice, I would recommend the Virtual Desk Top (with VLC) solution as being the most likely to deliver the highest quality output, with the fullest set of functionality for the reasons below:
- Virtual Desktop with VLC, in addition to BigScreen and Pigasus, also supports chapters, so you can choose the point at which you wish to start watching the video in 3D and so return to where you left off. See example screen shot below, showing chapter links at 10 minute intervals:
- UMS adjusts the bit rate of the video based on the performance of the renderer client over the network. If the renderer client is VLC and UMS is running on the same high performance machine (which would be the standard case), then this rate will be very high (as there is no network in between VLC and UMS) and so this rate is maximised.
- The software/CPU based libx264 encoder (the default) is able to deliver a very high quality output when a high bit rate is available.
- Virtual Desktop has limited 3D output format support, but importantly the high resolution 3D Full Side-by-Side (SBS) format is supported. Virtual Desktop can support mirroring a desktop at up to at least 3840x2160. This would be sufficient to render a 1920x1080 video for both the left and right eye at full resolution. VD mirrors your desktop in the headset. Full SBS mode is the 3D output format that should be selected in the UMS 2D to 3D settings for Virtual Desk top. Note, this is NOT the default. The default is Half Top/Bottom. You should change this default to Full SBS (the first in the 3D Output Format list) if you are using Virtual Desktop with VLC.
- Virtual Desktop is very highly optimised at being able to deliver the desktop screen content to the VR headset at high frame rates (up to 120 FPS for the Quest 2 for example). One of its main use cases is to support gaming. This optimisation is much more efficient than sending the compressed 3D video over the network, as would be the case for the DLNA media clients on the network, which then need to decompress the video before it can be displayed.
The article below includes a demonstration video showing how to connect VLC to UMS as well as a demonstration of the 3D video output but first a change, described below, is needed to the UMS "VLC-for-desktop" renderer configuration file in order to enable 2D to 3D for this renderer client:
The "VLC-for-desktop" renderer configuration file requires a minor addition to enable 2D to 3D support. To do this, go to the folder:
C:\Program Files (x86)\Universal Media Server\renderersOpen the file below in notepad:
VLC-for-desktop.confAdd this line to the end of the file:
AviSynth2Dto3D = trueA slight complication is that the "renderers" folder is write protected, so you will have to save the updated .conf file to another folder first before copying it back to the original location (windows will prompt you for permission to copy and overwrite the original file when you cut and paste the file). Remember also to restart UMS for the change to take effect.
If you want to add additional renderers, use the "SkyBoxVRPlayer.conf" renderer config file as an example and review the guidance given in the "DefaultRenderer.conf" file for configurable settings.
For background and further reading on the 2D to 3D algorithm, see the article below:
