Project Orion is Meta’s first pair of true AR glasses. They project hologram onto the world and are controlled with a neural wristband. They aren’t for sale but they’re a prototype of what Meta thinks will replace your smartphone in the coming years.
Category
🤖
TechTranscript
00:00This is Orion, Meta's first pair of AR glasses.
00:12Up top, I want to be clear.
00:13Orion was supposed to be a product you could buy.
00:16But it's not.
00:17Instead, it's a peek at what Meta has coming for hardware it's releasing over the next
00:21few years.
00:22And in a bigger way, it's a marker of where the smart glasses of today are eventually
00:27headed.
00:28I was one of the first people outside of Meta to try Orion.
00:30For about two hours, I used the glasses to browse the web, play games, and video call
00:35with people through Messenger.
00:37Oh, and Mark Zuckerberg beat me in Pong.
00:39This is going to be the whole interview.
00:43It's just an hour of us.
00:44It's like, we're going to ask questions, but instead I just...
00:45You say you're undefeated?
00:46No, no, no.
00:47Okay.
00:48Absolutely not.
00:49I'd never won a game before this week.
00:50Okay.
00:51Oh, okay.
00:52I'd only been defeated.
00:53Good game.
00:54You have an unfair advantage, but that's all right.
00:55I've played a few more.
01:02Orion exists in three parts.
01:04The glasses themselves, a neural wristband for controlling them with finger movements,
01:08and a wireless compute puck that offloads app logic from the glasses to help improve
01:13battery and reduce heat.
01:14The way you navigate the interface is through Eye and Hand Tracking, Voice and Access Control.
01:19The way you navigate the interface is through eye and hand tracking, voice, and most importantly,
01:24the neural wristband. It's hard to compare this experience to a laptop or phone,
01:28but in the simplest of terms, your eyes or hands act as the mouse, and you click by pinching your
01:33fingers together. If you've ever used the Vision Pro, you'll get how the eye tracking works pretty
01:38quickly. I want to focus on the wristband for a second because it's truly one of the most magical
01:42experiences I've had with a piece of new technology, and I think the world will be seeing it
01:46in the wild even before the commercial successor to Orion is released. It's about the size of a
01:52Fitbit and uses electromyography to interpret neural signals and translate them into the input
01:58for the glasses. The main gestures it recognizes are pinching your index finger with your thumb
02:02to select, pinching your middle finger and thumb to invoke the main menu, and swiping up or down
02:08with your thumb against your closed palm to scroll. There's haptic feedback on the band to
02:13let you know when it recognizes a gesture, which is a helpful signal that made me quickly
02:17comfortable with using it. Do you like using hand tracking at all with this, or do you mostly just
02:22do the band? It depends on what the app is. I mean, for some of the things, I think it is still pretty
02:27natural to like reach out and touch it and all that, but I don't know. I mean, the band is just
02:32going to keep getting better and better, and right now you have to do these little subtle gestures,
02:36but I mean, the plan is over the next few years to make it so you can just pick up almost motionless
02:42gestures that you make, so I think that'll be pretty wild. Yeah, the first thing you notice
02:46when you put Orion on is its 70 degree field of view, which is much wider than any pair of
02:51AR glasses I've tried to date. A narrow field of view means that AR graphics don't fill much of
02:57what you're seeing, but with Orion, I had to get up pretty close to something before its edges
03:01started to disappear. This made a huge difference for the overall experience. ULED projectors inside
03:07the frame of Orion beam graphics in front of your vision via wave guides in the lenses, which are
03:13made of silicon carbide and not glass. Seven cameras and sensors in the frames sense the world around
03:19you to anchor virtual objects in space. You can leave a window open, turn your head and walk away,
03:24come back, and it's still there. The quality of the graphics isn't at the level that I'd want to
03:29watch a movie in the glasses, but I had no problem reading text on a web page that was several feet
03:34away. My demo consisted of experiences designed to simulate how one may use AR glasses throughout
03:40the day in the future. It's clear Meta has an idea for how people will use glasses like these,
03:45but the software is early. My demo was very much on rails and guided by Meta employees in the room,
03:52though I was able to navigate the glasses on my own a little bit and jump in and out of apps.
03:56The coolest part of the demo was using Meta AI to identify ingredients for a smoothie that were
04:00out on a table and then have the glasses show me a recipe to make them with instructions.
04:05It was a peek at how generative AI can intersect with a form factor like this
04:09in pretty powerful ways. Do you guys know that I make smoothies like every morning?
04:14Like this is kind of getting creepy. I'm not gonna lie.
04:18Good to see you man. Yeah. Give me a smoothie recipe.
04:24Sure, let me take a look at what ingredients you already have. Then I can put a recipe together for you.
04:30I found several ingredients. Let me make a recipe. That's all right. It missed the pineapple,
04:35but that's okay. You don't really want that. Yeah, deep down. Okay, here we go.
04:41Matcha banana boost incorporates most of your ingredients. I would do that. I've never done
04:45matcha in a smoothie. We also chatted with our editor in chief, Nilay Patel. Can you see me at
04:50all? What do you see? I see nothing. I see gray, gray screen. That's what Alex looks like. I see
04:57you, but I've got verge pulled up on a pain here. And then you're just like a video
05:02hanging out in the middle. Do you see like a whole square? Am I just like floating as just my head?
05:07Yeah, you're floating. It's like a square window. You're floating and I can like come up to you.
05:11What's the resolution like? I just came off the verge cast. I got like a very French
05:15headphone hair going on here. Can you see that? It's good. It's not about good. It's something.
05:19I mean, it's like, it's, you know, it's not like you're going to like to watch avatar,
05:25but it's like, it's good. Like I can see you. Well, you sound great. I have to say,
05:30I can't tell if that is the microphone on the glasses or the speakers on the iPhone,
05:33but you sound great. It's the glasses. Yeah. It's weird that I can't see you. Are they going
05:38to add avatars to this thing? Yeah. So the glasses have inward facing cameras, so they will scan
05:44my face and represent me as a Kodak avatar when I'm calling you and they're not,
05:48but they're not selling this thing. Yeah. No, they're not. They didn't make many.
05:51They cost a lot of money. Amazing. We'll tell them to send me one. Yeah. Yeah. Mark Zuckerberg
06:05has been talking about Eric glasses for a long, long time. He's called them the Holy grail device
06:10that will eventually replace smartphones. And Orion is the first pair of Eric glasses I've
06:15tried that made me feel like that future isn't so far away that it feels like a pipe dream.
06:20But there was a moment in my demo that showed me just how much work meta still has to do to get
06:26this right. At one point I had multiple windows open to the side while sitting across from someone
06:31at a table and an incoming video call then pushed one of the windows directly over the person I was
06:37sitting across. And at that point it felt like the glasses weren't augmenting reality, but breaking
06:42it. Orion was going to be a product meta sold to the world until a couple of years ago when it
06:51realized it couldn't manufacture them in a way that would even make sense at an ultra high end
06:56price point. So instead it's made about a thousand of them for internal prototyping and demos like
07:02the one I received. So why is meta showing Orion to the world? If you can't buy a pair, it's
07:08impressive even for a prototype. And it's not an exaggeration to say these glasses are the
07:14end state of Zuckerberg's big bet on the metaverse. After all these years and the billions of dollars
07:19poured into making them, I think the company wants to have something to show off even if
07:24AR glasses still aren't quite ready for primetime. I do like the visual of this looking like
07:31Mime Warners. Yeah, well this is like one of the oldest video games ever,
07:38right? So I guess it's appropriate we're doing it on here. But we got some more Dimensionals.