Meta is skipping ahead of Apple at implementing AI in VR. While Bloomberg’s Mark Gurman previously reported that Apple Vision Pro will get Apple Intelligence sometime after this year, Meta’s Quest headsets will get AI next month.
Based on the Meta AI Assistant already baked into Ray-Ban smart glasses, Meta AI with Vision on Meta Quest will allow users to ask Meta questions based on what they can see with the full-color passthrough cameras on their device. That means it will be limited to Quest 3 and Quest Pro devices at first, although Meta did hint the AI might eventually also be able to detect objects seen on the Quest 2’s more basic sensors as well.
Meta says users could ask the AI to help them complete an outfit just by looking at it, and released a video where the headset was able to identify a dog even as it was running in and out of frame. The AI can also answer more general, non-vision based questions, like suggesting good restaurants nearby or helping bring up the most famous quotes from Hamlet. Meta AI can additionally use Bing to search live data from the internet, so it’s not restricted to its training data.
Meta AI with Vision relies on multimodal technology, meaning it can accept inputs from multiple mediums and translate content between them. The company recently said it would not be releasing its next multimodal model in the EU, so the Quest’s AI features will only be available in Canada and the United States first. They’ll also only support English to begin with.
Despite the limited initial release, the company is going all in on the AI assistant, saying “Meta AI will replace the current Voice Commands on Quest.” This means it will also be able to help you control your headset, although to what extent has yet to be clarified.
Meta AI will launch on Quest in experimental mode first. To try it out once it’s ready, open the settings menu on your headset and toggle it on under the “Experimental” submenu.
