Every VR developer needs to have a hand tracking plan now.
The winds are changing very quickly and not having a plan could potentially be a costly mistake. If you are developing a title that is controllers only, that needs to be a plan of intentionality.
For example, if you’ve decided there is a market for a fast-paced VR shooter and controllers are the only way to deliver that? Cool. But this decision needs to factor in that you will not be able to launch on an Apple device or any other device that follows their lead. Additionally, this will be a harder sell over time on platforms with controllers as they adopt hand tracking as their primary input. It would be a really bad plan to start developing a controller-only VR game just because that is what has been done before.
Hand tracking will become the default and primary mode of interaction in VR. This process has already started with Quest. I use controllers less and less and hand tracking, combined with eye tracking, is the only input for Apple Vision Pro. Apple’s headset will be a huge push in the direction of hand tracking.
The shift to hand tracking will be quicker than most think and leave a lot of quality titles that didn’t plan ahead in a bad spot, especially with development cycles in the 2-3 year range. Once users start using hand tracking, some will have limited desire to switch back to controllers. For me, it’s already bad enough that I’m frequently misplacing my controllers.
Some developers will play the wait and see approach. The potential issue with this is that hand tracking is very hard to retrofit and, if you have a button-heavy VR design, it’s even harder. Trying to do this post-launch may not be compatible with your existing controller specific design, could yield a worse product, or just be too expensive to develop. One of the huge advantages of working with hand tracking at the beginning is designing the product for the unique advantages and constraints hand tracking brings. There are things that you can do with hand tracking that controllers could only dream of doing and we will see that in coming months and years.
One big challenge for developers that are skeptical about hand tracking (I know I initially was) is what they have seen to date.
Some developers I talk to have a lot of negative things to say about it because they are early adopters. VR developers kind of need to be early adopters, but that means a lot of their experiences were with poor early versions of hand tracking (often gesture based) that are not reflective of the state today.
Meanwhile, other developers that have gone out of their way to expose themselves to more recent hand tracking applications have come away underwhelmed. The issue here is what they are seeing, almost universally, is shorter development cycle demos or ports into existing games. This can help paint a picture of possibility. Early VR developers know the first couple years of VR was rough for quality because there wasn’t much knowledge to pull from and there were so many ports of pancake games. You had to look past some of that to see where things were going. As someone that has been working almost exclusively with hand tracking for the last 2+ years, I can tell you the state of the art is a huge leap from what is visible now. Combine that with the constant and continuous improvement to the quality of the underlying hand tracking? The landscape in 2 years is going to look quite different.
This shift to hand tracking as the primary input is great news for VR. This will make experiences more intuitive, approachable and reduce overall VR friction. It also has potential to bring down costs (ditching controllers) and attract more mainstream users. As a developer, until you have worked exclusively with hand tracking you don’t realize how much real friction controllers add.
This isn’t to say controller-based VR games will go away. Some people use a gamepad for their smartphones, after all. Over time, though, dedicated tracked controllers will become more and more niche. As a developer, particularly a small one, being in a niche can often be a wise decision. VR still very much is this, but being in a niche of a niche can get dicey if you aren’t careful.
Devin Reimer was CEO and CTO at Owlchemy Labs. He led technical development of Job Simulator and subsequent VR games before founding AstroBeam, a studio working on multiplayer VR experiences.
