Project Overview
A VR research project in music performance led by Professor Lance LaDuke at Carnegie Mellon University. We aim to recreate concert halls and theaters in a VR environment where music students may practice to overcome performance anxiety.
Position: VR Developer, World Designer, Programmer
Creating the Environment
When I joined the research team in the Fall of 2021, the previous team members had created a 3D version of the Kresge Recital Hall in CAD, using approximate dimensions calculated manually.
Given the inaccuracy of the dimensions and textures, I decided to explore the possibility of using photogrammetry to create more realistic and accurate 3D version of the Kresge Recital Hall.
Using Metashape, I aligned around 150 photographs of the recital hall to create a very rough render.
The photographs were taken with a smartphone, which created artefacts and noise under low-light conditions. This may have contributed to the uneven and bumpy surfaces.
A solution may be to take photographs with a DSLR, with a fixed f stop, shutter speed and ISO settings, maintaining the consistency of the photographs' exposures levels, color balance and brightness.
I imported the model and the texture into the Unreal Engine, which our VR Project is built in. Despite the harsh vertices and noise, the lighting and the accurate dimensions effectively capture the atmosphere of the recital hall. A more refined mesh will most likely improve the realism of the setting.
The former 3D CAD model is much more refined, with clean surfaces and smooth finish. However, the dimensions are inaccurate, and the details around the pillars and the walls.
Given the benefits and the trade-offs, I am currently looking into baking on photo-realistic textures onto a low-poly models as seen in the 3D CAD model. This can not only make it more realistic and aesthetically pleasing, but also optimize for mobile platforms such as the Oculus Quest.
