Google Light Camera turns VR into art

Google Light Camera turns VR into art

photo: technologyreview

According to the developers, the new VR application will allow you to explore new worlds with amazing depth and detail of images.

I just visited the cockpit of the NASA space shuttle, strolled through a chic mosaic house in California, and peered through decorative stained glass windows into an old chapel – all without leaving my chair!

In fact, this chair is located in the Google headquarters in Silicon Valley and on my face is a virtual reality headset from HTC Vive.

But due to the fact that the VR images were shot in high resolution on a prototype that is able to capture the depth of objects in the real world, I felt more in the real world than with any other VR. And it was damn cool! I could look around in the space shuttle, press buttons and switches on the walls of the cockpit that were previously inaccessible to me! And in the chapel, I watched the glare of the sun, which moved with me! – one of the developers who experienced the new reality of Google shares his impressions.

When it comes to a static environment within virtual reality in which users can move around, the right lighting can make an image both more believable and completely disrupt the balance of reality.

Google recently launched a new free app for VR devices, aimed at allowing users to experience the “light areas” of scenes of the world around them.

In fact, the highlights in an image are how the lighting looks from different angles. If you look at the screen of your smartphone, it is the reflection of the light that will make its perception realistic. Yes, most physical objects do not reflect the surrounding world with the accuracy of a mirror, but even objects such as skin can change their texture, depending on the “angle of view”, – written in the blog of the project developers.

The new app will be available through the Steam video game platform, and is designed to work with the HTC Vive, Oculus Rift and Windows headsets for mixed reality.

To recreate a realistic perspective between objects, Google uses multiple cameras with different viewpoints at once in order to be able to capture as many areas of the scene as possible. This gives you VR objects that move with you or in your field of view that shifts with head movements.

Google camera captures the cockpit of the space shuttle Discovery.Google camera captures the cockpit of the space shuttle Discovery.

Now, if you want to know more about how the new Google app “sees”, then Welcome to the Light Fields is where we are ready to educate users on what exactly the prototype is and how it helps make the virtual reality experience even more realistic. …

As the name suggests, images in the app are created by recording and recreating the light fields of scenes in which users can walk around and watch the lighting change these environments.

In prototyping, Google used several small, interconnected cameras to capture many images, as if you were looking at an object from different points of a small sphere (about the size of a beach ball). Taken together, the resulting photographs create a spherical patchwork quilt that reproduces with amazing accuracy the rays of light that a viewer might observe in this particular confined space.

A light field camera outside a mosaic house in Venice, California.A light field camera outside a mosaic house in Venice, California.

And although the application includes only static footage, not video, it already gives an idea that virtual exploration of historical sites and works of art, as well as the possibility of creating hyper-realistic VR films, is just around the corner.

If the work on capturing images of the real world is carried out well, then we will really give you a teleportation machine that can take you anywhere, anytime! Says Paul Debevek, Senior Scientist at Daydream’s Virtual Reality Group.

Paul Debevec (fl: YouTube)Paul Debevec (fl: YouTube)

But so far, this idea is mostly just a promise. For many reasons, the technology isn’t very popular with consumers: bulky headsets and expensive hardware don’t always give a real VR experience.

Debevec believes that light fields could potentially solve VR’s “implausibility” problem. Thanks to the light camera, Google could become the first company to reproduce motion parallax (objects that are nearby move more, and those that are in the distance less) in an application accessible to all.

To create a light camera, the developers had to modify an existing setup consisting of an axis with 16 connected GoPro cameras. But instead of creating a spherical base like some other developers (including Facebook), they connected the cameras in a vertical arc and placed it all on a tripod with a motor and battery. Thus, the device could slowly rotate in a circle to widen the capture angle of pictures of the real world.

photo: TechCrunchphoto: TechCrunch

A typical circle takes about 30 seconds, although the device can rotate more slowly to take better pictures in low light, explains Devebec.

But the question remains open – how best to “place” people in these static scenes?

In one of the scenes I ran into people and this meeting was quite creepy. A man was sitting motionless in the chapel, but at the same time he moved his eyes, which created the effect that he was watching me, – the researcher shares his impressions.

Despite the small nuances, the performance of the new Google camera is just fine. And evaluating what you can already see with a VR headset, it becomes obvious that the virtual world is already much closer to reality.

Go to our cases Get a free quote