visionOS 2.4 will bring Apple Intelligence to Vision Pro, a Spatial Gallery app, an iPhone app for remote installs, and a new iPhone/iPad-driven guest flow.
The last significant visionOS update was 2.2 in December, which brought the Wide and Ultrawide Mac Virtual Display modes. visionOS 2.3 instead focused on bug fixes and security updates.
The first visionOS 2.4 developer beta is available to install now, and Apple says the stable release will arrive in April.
How To Install visionOS Developer Betas
To enable downloading beta Apple operating systems you just need to sign in with your Apple ID on the Apple Developer Center website at least once. You don’t need to join the paid Apple Developer Program, but you will need to accept the terms of the Apple Developer Agreement.
Then, in the headset, navigate to Settings -> General -> Software Update -> Beta Updates and select “visionOS 2 Developer Beta”.
Note that installing a beta version of an operating system is only recommended if you’re willing to accept bugs, instability, and the small chance it could put your device in a state requiring a factory reset. Some apps may even stop working. In exchange, you’ll get to try out the new features and improvements in advance.
Apple Intelligence
Apple Intelligence is the company’s name for its generative AI features. Some features run on-device, while others are offloaded to the company’s Private Cloud Compute (PCC) servers, and certain requests reach out to OpenAI’s ChatGPT, if you give permission for that.
Apple Intelligence arrived on iPhones (15 Pro and later), iPads (M1 or A17 Pro and later), and Macs (M-series chips only) late last year, and Apple didn’t say at the time whether Vision Pro would get it too.
With visionOS 2.4 Apple is bringing the following Apple Intelligence features to Vision Pro: Priority Notifications, Notification Summaries, Smart Reply, Memory Movie creation and Natural Language Search in Photos, Priority Messages & Mail Summaries In Mail, Writing Tools, Image Wand in Notes, Genmoji, and Image Playground.
At launch, Apple Intelligence on Vision Pro will only support US English, with support for “additional languages” coming later this year.
Priority Notifications & Notification Summaries
Apple Intelligence will automatically decide which notifications are most important, such as messages from loved ones about imminent events, and surface them at the top of the notification stack.
Further, longer notifications and groups of notifications from the same apps will show an AI-generated summary in place of its content. We should note that this feature has been widely criticized for sometimes misconstruing the content of notifications, however.
Tapping the notification summary will bring up the original notification.
Smart Reply In Messages & Mail
Smart Reply “identifies questions and suggests relevant replies” to messages and emails, which Apple says will let you “easily respond to texts and emails with just a few taps” on Vision Pro.
This feature should be particularly useful for XR, for when you don’t have a Bluetooth keyboard connected at least, since floating virtual keyboards are slower to type on than touchscreen keyboards.
Memory Movies & Natural Language Search In Photos
In the Photos visionOS app, Apple Intelligence will bring the ability to generate a Memory Movie, as well as more advanced search.
“Simply type a description, and Apple Intelligence will pick out the best photos and videos, craft a storyline with chapters based on themes identified from the photos, and arrange them into a movie with its own narrative arc and a soundtrack,” Apple explains.
0:00
Meanwhile, the Natural Language Search feature promises to let you find photos and videos in your library by just describing them.
Priority Messages & Mail Summaries In Mail
Just like with notifications, in the visionOS Mail app Apple Intelligence should surface the most salient messages at the top of the stack.
Each email in the list will have an AI-generated short summary of its content, instead of just the first line as in traditional email clients.
Further, upon opening an email you can choose to AI generate a medium-length summary of its content.
Writing Tools
The Writing Tools feature will offer four kinds of tools for “rewriting, proofreading, and summarizing” long-form text that you type or dictate “nearly everywhere” in visionOS, including Mail, Notes, and many third-party apps.
These tools are:
- Proofread: “checks grammar, word choice, and sentence structure with suggested edits”.
- Rewrite: “adjust the tone of text to make it more friendly, professional, or concise”, or use Describe Your Change to specify the exact kind of change you want.
- Summarize: recap the text, break it out into Key Points, or turn it into a bullet point list or table.
- Compose: leverages OpenAI’s ChatGPT to generate any kind of textual content.
Image Wand In Notes
Image Wand in the visionOS Notes app will let you “create images based on rough sketches you create”, or AI generate an entirely new image “based on words and images from the surrounding area”.
Genmoji In iMessage
Genmoji is Apple’s feature to generate your own custom emoji in iMessage by typing or speaking a description of it.
0:00
Within iMessage, these Genmoji can “be added inline to messages, shared as a sticker, or sent as a Tapback”.
Image Playground
Image Playground lets you AI generate “fun and unique images from themes, costumes, accessories, and places”.
0:00
According to Apple, “users can add their own text descriptions, and can even create images in the likeness of a family member or friend using photos from their photo library.”
Image Playground is integrated into apps like Messages and Freeform, and is also available as a new standalone visionOS app.
Spatial Gallery
Spatial Gallery is a new visionOS app from Apple that “features a curated collection of spatial photos, spatial videos, and panoramas from artists, filmmakers, photographers, and more.”
At launch, Apple says it will offer “remarkable perspectives from photographers like Jonpaul Douglass and Samba Diop; new stories and experiences from iconic brands including Cirque du Soleil, Red Bull, and Porsche; behind-the-scenes moments from Apple Originals like Disclaimer, Severance, and Shrinking; and special moments from top artists.”
0:00
Rather than being an open platform like YouTube or Vimeo, which added spatial video support in October, Apple’s Spatial Gallery sounds like a highly curated closed platform, offering quality over quantity.
Spatial Gallery will be available in all Apple Vision Pro countries except for mainland China.
Apple Vision Pro iPhone App
Apple says the new Apple Vision Pro app for iPhone will let you remotely queue apps to download & install to your headset, see device information, and browse curated recommendations of the best visionOS content.
The My Vision Pro interface will show device tips, the current visionOS version, and the serial number, and will let you set up Personalized Spatial Audio by scanning your face shape with your iPhone’s TrueDepth sensor.
Meanwhile, Apple says the Discovery interface will show “popular apps and games on the App Store; nearly 300 3D movies, Apple Immersive titles, and more video content on the Apple TV app; and the latest spatial photos, spatial videos, and panoramas featured in the Spatial Gallery”.
The Apple Vision Pro app will be part of iOS 18.4, and can also be downloaded from the iOS App Store.
iPhone/iPad-Driven Guest User Mode
Currently, initializing Guest User mode on Vision Pro requires putting on the headset and unlocking it first, and if there are any issues you need to adjust for after the guest puts the headset on, you need to do that all over again. From experience, this can lead to frustration.
With visionOS 2.4, Apple is adding the ability to approve Guest User mode from a nearby iPhone or iPad signed into the same Apple Account.
0:00
As with the in-headset Guest User mode interface, this iPhone/iPad interface will let you choose which apps the guest has access to, as well as initiate View Mirroring with AirPlay.
Editor’s Note: Article updated briefly after publication to remove mention of Keynote, which Apple removed from draft press materials.
