Palmer Luckey described how Anduril’s Eagle Eye helmet will give soldiers superhuman senses, describing it as “by far the best AR/VR/MR vision augmentation system that has ever been built”.
Last week, Anduril announced that it is taking over the US Army’s IVAS program from Microsoft. Anduril is the defence company Luckey founded after being fired from Oculus, currently valued at $28 billion.
That initial announcement didn’t come with any specific details on the hardware or capabilities of Anduril’s IVAS solution. But in an interview with former Navy SEAL Shawn Ryan, Luckey revealed the first details of the system as well as its name: Eagle Eye.
David Heaney
Luckey told Ryan that Anduril has been “investing a ton of resources” into the development of Eagle Eye for “years” now, long before it knew it could secure the IVAS contract. A defining tenet of Anduril has been to develop products in advance of offering them to the US government for a fixed price, a stark departure from the approach of legacy defense contractors, who secure contracts to develop hardware with the ability to go over budget.
Unlike Microsoft’s HoloLens IVAS, Anduril’s Eagle Eye is an “integrated ballistic shell”, not something you strap onto existing helmets. The strap-on approach was cumbersome, Luckey explains, leading to imbalances and “snag hazard”.
“[With Microsoft’s IVAS] you run a strap to the back, there’s a big battery pack and compute module back there, there’s a big sensor brim you clip onto the helmet. The problem when you’re trying to clip on to an existing system is it ends up not very tightly integrated, lots of snag points, lots of snag hazard, it ends up being very heavy and misbalanced where it’s really torquing your neck.
The thing that I’m building is an all up integrated ballistic shell that integrates hearing protection, hearing augmentation, vision protection, vision augmentation, all into one seamless ballistic shell that protects you from airbursts, direct fire rounds, blast and concussion – the whole thing in one integrated, seamless product.”
As for that “vision augmentation”, Luckey claims Eagle Eye will be “by far the best AR/VR/MR vision augmentation system that has ever been built”, with resolution, field of view, and sensors superior to anything else.
“It is going to be by far the best AR, VR, MR vision augmentation system that has ever been built; in terms of resolution, in terms of field of view, in terms of graphical fidelity, in terms of sensor quality, and what you can do with those sensors.
It is a bigger jump from what exists today than the jump that I made when I started Oculus. It is a jump that I think cannot be overstated.”
On the software side, Eagle Eye will leverage Lattice, Anduril’s original and primary product. Lattice is, at its core, a distributed software system that takes in sensor data from a wide variety of military platforms, including both Anduril and third party assets, and autonomously integrates it to build a unified view of the entire battlespace, while bringing attention to the most salient targets.
So what can Lattice running on Eagle Eye do for soldiers?
The first trial Anduril conducted, when it was only set to have Lattice running on HoloLens IVAS, gave soldiers warning of incoming drones, and guidance of how to get to safety from them.
“I can see drones that are coming to attack me. I can see what their attack vectors are. I can see where the people controlling them are. I can see where I need to go to be safe in an amount of time that is reasonable before that drone actually gets to me. Really, really powerful stuff.”
Another use case Luckey mentions is marking targets, without the need to use smoke or a laser that enemies can see, as well as the system automatically sharing the position of visible enemies, highlighting them for other soldiers even when out of sight.
“If you want to mark a target for somebody else, you’d have to use a laser, right? What if you could mark it digitally, and you don’t omit any signature for anyone else with [night vision] to see? And what if anyone else can see it?
Here’s an even crazier [capability] – and when I say crazy, I don’t mean like a hypothetical, you could just do this with this tech. Imagine that you’re doing some kind of pincer, and you have some guys over there, some guys over there, and I can’t see a guy behind that building, but you can. If you’re seeing him, it’s taking that track, it’s taking that enemy mark, and it’s now putting it into my vision.
I can now see through the building, through the wall, and I can see the guy coming around the corner before he’s there.”
Because Lattice integrates with a wide range of military platforms, including drones, the source of this kind of shared situational awareness isn’t limited to other Eagle Eye wearers.
“And it’s not just the helmets. You’re also anything that’s seamed into Lattice. So if there’s a drone overhead that sees a guy five clicks out that way coming up a hill ,and he thinks he’s going to set up on top of that hill and pop you, imagine if it sees him, it notifies you, you look at him coming over that hill and you bring your rifle on target.
Literally the moment he clears. you’re taking out that threat. That’s the type of combined sensing and combined arms tactics that this enables.”
And unlike when limited to human intelligence, Lattice can keep track of an effectively unlimited number of potential threats, leveraging all available sensors in the vicinity.
“When you have computers watching all these sensors, and you have an unlimited amount of attention that a computer can give, you can have that computer doing everything for you. You can say, ‘watch every single one of those windows, and watch every single one of those doors, and just let me know if I need to worry about it.'”
Beyond just awareness, Luckey wants Lattice on Eagle Eye to offer proactive guidance to soldiers, leveraging predictive AI to help save their lives in battle.
“Imagine if it can look at an aircraft that’s coming to engage us, and the AI has the expertise of the world’s best fighter pilots. Isn’t that extraordinary? Or the best rotary wing pilots. By building all this expertise in, you’re potentially building a guardian angel that is this superhuman, superintelligent thing.
Imagine if you look at it and it says: ‘oh shit, that’s a Russian pilot’. Based on the tactics that it’s observing or the way that it’s working, imagine it says: ‘oh, he’s going to come back around to do a low attack run’.
This stuff is way beyond just ‘oh, there’s a dot and there’s the enemy’. If you have the world’s best expertise of the world’s best warfighters distilled into one super AI, it’s like you’re rolling around with a whole team of experts sitting on your shoulder telling you what might happen next.”
Further, the Lattice AI would answer questions about what the soldier is seeing, leveraging the same kind of multimodal AI used for civilian purposes in the Ray-Ban Meta glasses and phone apps like ChatGPT.
“What if you could say: ‘hey, how do I defuse this bomb?’ Or imagine like seeing a vehicle and being like: ‘How do I hotwire this car?'”
Later in the interview, Luckey muses on the issue of whether to sell Eagle Eye to civilians, not just the military. He says that while in principle as a libertarian weapons enthusiast he would want to, in practice this would risk falling into the hands of hostile nations.
“Anything that I sell to civilians, I’m effectively also selling to Chinese special forces, to Russian commandos. Now, I’m not saying that they’ll be able to get enough to outfit their whole force, but anything you sell to civilians is eventually going to get sold to a traitorous American.”
One non-military group that Luckey does seem willing to sell a Lattice-powered head mounted display to though is law enforcement. While he doesn’t think a bulky Eagle Eye helmet would be appropriate for “most” police, given their need to interact with people face to face, he does see a place for a less advanced product that would “look more like a pair of Oakleys” than a “RoboCop helmet”.
“That is still running lattice. It’s still showing them where threats are. But it probably doesn’t need the ability to pick up an attack helicopter 15 clicks out and then flag them where they need to go.”
“I want something that’s watching my six. I want something that’s watching for threats that I’m not seeing. I want basically a guardian angel on my shoulder that is able to do what backup would normally do for me, and at a superhuman level.
Imagine if you could have not just eyes in the back of your head, but what if you could have a hundred eyes all throughout your head all looking out into the world and at the slightest disturbance be like: ‘holy shit, I think someone’s opening that window and firing, they’re they’re aiming a gun from that window’.
It should be telling me not ‘hey, you know, be aware that someone might be shooting you’, it should be giving you even more direct commands than that, like, it should throw a red threat alert and show you a direction you need to throw yourself immediately to not get shot, to get behind cover.”
For a full description of Eagle Eye, as well as a first hand account of Luckey’s firing by Facebook, influence on PlayStation VR, and views on robotics, exoskeletons, military contractors, China, and more, I recommend watching or listening to the full interview.
