Apple’s 2022 WWDC conference is currently underway. In the keynote on 6th June we heard the headline announcements about iOS 16, iPadOS 16, watchOS 9, and macOS 13 (Ventura), along with the news about the new baseline M2 processor inside a redesigned MacBook Air and 13″ MacBook Pro. The Apple XR headset didn’t directly make an appearance, but we could feel its future presence in other ways.
Away from the headlines, XR developers were abuzz on Twitter as they read through the Apple Developers notes – not just ARKit and RealityKit, but also improvements to Metal too. And as some of the other announcements were discussed and critiqued, the pennies started dropping in other areas too. A headset can’t be far away after all.
ARKit 6
ARKit 6 has improved the resolution of the video feed used to present 3D models in the AR viewer. This has now risen from HD to 4K with HDR, meaning more realistic experiences are possible and AR footage can now be captured for other uses in much higher quality.
High resolution stills can also be generated.
Instant AR sessions will be available on iPhones and iPads with LiDAR sensors, so no more waving the phone about to define a suitable surface.
Other improvements include capturing photos at full camera resolution during an ARKit session (I didn’t know that it couldn’t before?), provision of EXIF data in the photos (useful for post-processing), and capture of an uninterrupted video stream an an AR session (hopefully this will mean less choppy AR video captures).
Apple have made a useful video on ARKit 6 with code samples available.
RoomPlan is a new Swift API that uses the camera and LiDAR sensor to create quick scaled plans of rooms and the furniture within them. It creates a small isometric plan of the room as you move around it, using AI to guess the kind of furniture. Jonathan Stephens called the sample app “Tron-like” as it uses bright glowing lines to delineate edges. I would imagine that this tool would be very useful for a forthcoming headset, as well as for the obvious planning of rooms.
RealityKit
In 2022 RealityKit adds some nice new features. There are now custom shaders, bringing physically based materials to 3D assets. It also enables environmental reflections, ground shadows, camera noise and motion blur to help virtual content become “nearly indistinguishable from reality” (their words, not mine!) in AR apps.
Metal 3
The Metal 3 API allows renders to use a lower resolution and frame rate, then be upscaled and smoothed without (apparently!) any performance or quality hit. If the new M2 processor (or mobile A series equivalent) were to be powering a forthcoming headset, then this processor and this technique will certainly make an appearance to create great visuals with the least amount of heat (maximising battery life). This will be useful in so many apps – not just games.
There’s (probably) more
WWDC runs for a whole week, with new sessions on different topics each day. We will doubtless see some more features relevant to XR in the coming days. Take a look at the list of developer videos to browse through all the topics.
So whilst we haven’t yet seen an Apple XR headset, there’s a lot of ground-laying going on for when it is released. And of course, thanks to a slip-up in Apple’s own sample code, we know that there will eventually be a dedicated OS – realityOS – for it all to run on.
All in good time.