After years of speculation, Apple is getting into VR and AR in a big way. At its WWDC 2023 developer conference keynote, the company revealed Vision Pro, an XR headset combining both virtual and augmented reality. It's Apple's first big hardware category since the Apple Watch was released in 2015.

The headset has a single piece of laminated glass with a "wide array of cameras and sensors." The frame includes a button for photos and videos, and a digital camera to switch between levels of AR and VR. Vision Pro's frame has a thermal design that draws air through holes in the system to keep it cool. Needless to say, this isn't fanless.

Apple is calling it "the most advanced personal electronics device over" and is based on over 5,000 patents.

The light seal comes in multiple sizes and conforms to your face. Straps reach out from the frame with "Audio Pods" to deliver spatial audio. The headband is a ribbed, 3D pressure that stretches with an attachable mechanism. An adjustment dial lets you make small adjustments during use.

Glasses-wearers will use inserts that attach to the lenses. These are made by Zeiss.

Apple is promising two hours of battery on a charge (but can be used longer while plugged in). The hardware includes a battery pack that is wired to the headset, that you then put in a belt or in a pocket.

An external display will show your eyes so that people can recognize if you're immersed in VR or not, and keep people from feeling isolated. Apple hasn't released specs yet, but we'll update this article if we do.

Tech in Apple Vision Pro

Vision Pro will boast some impressive specs.  It fits 64 pixels in the space of an iPhone pixel. It has 23 million pixels across It uses two panels, for more than 4K resolution. It supports high dynamic range and allows for sharp text, Apple states.

Apple is using a dual-chip design with Vision Pro, with M2 and a new chip called R1. R1 is designed for input from 12 cameras, five sensors and, six microphones. M2 allows for performance, while R1 is designed for experiences to feel immersive.

Meta's Quest Pro has two LCD panels at 1800 x 1920 pixels with a claimed 106 degree horizontal field of view and a 96 degree vertical field of view. (Last week, Meta announced the Meta Quest 3, but didn't provide full specs ahead of a full launch later this year). An yes, there's a screen on the outside to show your eyes.

The Audio Pods trace your room with sound to customize it for you. Apple calls this "audio ray tracing." There are tons of scanners, including LiDAR, for 3D tracking. There's IR cameras and LEDs to track your eyes so you don't need "clumsy hardware controllers."

For FaceTime, Apple creates a digital version of you. After being scanned by the front cameras, Apple uses a neural network to make a 3D version of you to use in FaceTime calls. When someone in another Vision Pro is chatting with you, they'll see it in 3D.

The new OS is called visionOS, which supports iOS framework, a new real-time subsystem, a foveated rendering pipeline. and a multi-app 3D engine to keep multiple apps running in the same simulation.

For security, Apple is using Optic ID, a new take on features like Face ID and Touch ID. Your iris scan will be stored on device in the secure enclave. Apple says eye input will be separated to a special background process so that websites or apps won't be able to see where you're looking. Only finger taps will be registered.

Using Apple Vision Pro

Apple's operating system for AR and VR  will be somewhat familiar to users of iOS and iPadOS with a home screen with app icons and widgets that can be rearranged.
There's a home view in your space, with apps that appear like they float in your room. They respond to light and cast shadows on your home. You can scale apps and place them wherever in your space that you wish.

The switch between AR and VR is key here. Apple is relying on a mix between the two, so that people who use the headset don't feel entirely blocked out from the world. The dial, similar to the digital crown on the Apple Watch, lets you move in-between AR and VR.

The headset relies solely on your eyes, hands, and voice. There's no controller. Graphical elements on screen respond to your looking at them, and a tap of the fingers selects. You can flick your hands to scroll.

Apple is using the EyeSight moniker to show your eyes to others near you on an external screen. This, the company says, keeps you from being isolated from others and lets others know if you're immersed in VR or can see in AR.

For productivity, you get a "canvas for apps." You can place your apps anywhere in the screen. It looks like a floating version of Mac, with web browsing and other productivity apps around in in space. You can simply turn your head to move form app to app. You can even put 3D objects in front of you and see them from any angle.

Vision Pro works with the Magic Trackpad and Magic Keyboard, so you can use Bluetooth accessories. You can also use your Mac to put your Mac's screen on the headset for a massive 4K display.

There's also a version of FaceTime that lets you see digital versions of other people in one-on-one calls. When new people are added, they simply float in front of you. You can share apps with others while on a call.

Multimedia and Gaming on Vision Pro

Apple showed it off as a multimedia machine for use when you're not working. Photos float in mid-air, and panoramas taken from iPhone will wrap around you. You can take photos or video with the press of a button on the unit, to watch back in 3D later. Vision Pro blinks to let others see you're taking images.

Vision Pro also works as a personal movie theater. You can adjust the size in your space, and the OS automatically dims the surrounding area while adding spatial audio. You can even add an "environment" to work as a massive movie theater larger than your home. A specialized movie theater is meant to focus on film and nothing else. You can also watch 3D "spatial" movies; Apple showed off Avatar: The Way of the Water .

You can also use the Vision Pro to play Apple Arcade games with game controller support. Apple is promising hundreds of titles on launch day.

Bob Iger, the CEO of Apple, joined Apple CEO Tim Cook at WWDC, showing off Disney  Plus, including customized versions of shows like The Mandalorian, as well as customized ESPN game views that show replays in 3D. If also showed a Marvel: What If? game that seems to put you in the shoes of a superhero. Clearly, Apple has partners ready for this headset. Disney Plus will be ready on a launch day, Iger said.

Developing for Apple Vision Pro

Some developers have already had their hands on Vision Pro. Apple demoed a 3D beating heart as well as a design of an Alfa Romeo F1. These are similar to what we've seen on other headsets and even Microsoft's HoloLens.

But others, like a spatial interface for DJ-ing and a sky guide to make Vision Pro into a planetarium sound more unique. And Microsoft has its Office apps ready to go, as do as a bunch of video conferencing software companies.

The same frameworks in iOS and iPadOS are built into visionOS, meaning plenty of existing apps should largely work out of the box with the built in controls.

Apple has been working with Unity, and it says games and other apps based on Unity will be able to work with minor tweaks.

There will be a brand new App Store to see everything built for Vision Pro as well as a any other compatible apps.

Developing... refresh for more details.