The mouth tracking in VR: avatars become more realistic

The mouth tracking in VR: avatars become more realistic VR-the developer of the integrated pilot Lip from HTC Tracker ("the tracker's lips") in their social VR application and demonstrated that it is possible to do. Tomáš Mariančík under the name Frooxius is working on its application for virtual reality VR Neos in 2016, which he calls "creative social VR-sandbox". In it, users can play, build things, work or just spending time together. A feature of the platform is the wide support of hardware and software for tracking. The following video shows the tracking of the whole body by the eight trackers Vive. They are attached to the chest, elbows, hips, knees and feet. Finger movement can be read using the controller Valve Index, the movements of the eye — glasses HTC Vive Pro Eye. The result is a particularly expressive avatar.

HTC is experimenting with face tracking

However, Frooxius and this is a little. Currently the developer of a virtual reality working to support the Lip Tracker from HTC (tracker lips). In the spring of 2019 at GDC was announced Vive Pro Devkit, targeting developers and researchers. Frooxious shows the results of their experiments on Reddit and Twitter. The module captures more than the name implies: a developer shows his teeth, his tongue and inflates his cheeks, and it all passed in a virtual reality simulation in real-time. This makes it human or animal avatar even more realistic.

More social presence

Real and VR video recording was synchronized, but latency is as low as possible. The track is "quite reliable", if the module has a clear location opposite to the mouth area. The developer gets positive feedback since he started to use eye tracking and lips, the VR users began to refer to it.

"People like it when there is something funny, and I smile in virtual reality. Or they tell me that I went to sleep, when I yawn," writes Proxies on Reddit.

It is not known when the technology falls into the VR glasses. In any case, it should first be used in devices of B2B, such as Vive Pro Eye. NeosVR is still in early access and available for free on Steam. The article describes Steam Frooxius, what other features users expect in 2020. Source