ControllerPose: capture the full body with cameras on the controls
A group of researchers from Carnegie Mellon University (Pittsburgh) has presented a paper on the use of cameras in VR headset controllers with inside-out tracking for full-body capture. The team that forms Future Interfaces Group have called this method ControllerPose, by attaching cameras to the top and bottom of the Quest 2 viewfinder buttons.
Get to incorporate the whole body of our experiences in virtual reality without the need for trackers or external sensors was the purpose of this experiment, and that’s only with cameras that take viewers type Quest or the Peak is not possible, so a possible solution is the incorporation of cameras in the drivers. A similar technology will be incorporated by Project Cambria and Magic Leap 2.
The result of his research was a prototype, ControllerPose, which uses two fisheye cameras on each controller, one at the top and one at the bottom, which focus on the user. By means of a specific software, the field of vision of both is united and integrated into a single, wider one, capable of seeing most of the body. With this system, the posture of the entire body can be followed with an average error of the joints in 3D of 6.98 cm.
Future Interfaces Group is the same group of researchers that has designed an experimental accessory so that we can have haptic sensations in the mouth and lips area, as we showed in the Virtual Hour program on Sunday, May 1. The results of his research, both for feeling kisses in VR and on full body tracking, are available on his website.