Apple’s augmented / mixed / virtual reality headset is now almost ready for debut, but what an odyssey it was: as we have reported in recent days, a series of articles by The Information tells, in great detail, all the crucial stages of a project that began to take shape in the now distant 2015. The newspaper says it has spoken with several well-informed sources on the events, and a story of conflict of ideas on which direction to take for the project: on the one hand there was the division dedicated to mixed reality itself (called Technology Development Group), and on the other no less than Jony Ive, the most important and respected designer of the Apple, who for some years now has fired for self-employment.
The details that emerged in the series of articles are so many, but let’s try to do a schematic summary organizing the main points in chronological order.
- 2015: the genesis.
- April 1: Mike Rockwell is hired by Apple to lead the AR / VR initiative. He was formerly a vice president of Dolby.
- May 28: Apple buys Metaio. It is a German startup that deals with augmented reality, and was born about 12 years earlier in the Volkswagen house. Apple’s primary goal was to use some of Metaio’s technology for Project Titan, the self-driving car.
- spring / summer: Rockwell begins assembling an AR team with Peter Meier, co-founder of Metaio, and Fletcher Rothkopf, project manager for Apple Watch.
- 2016:
- First demos. Several augmented reality demonstrations are performed to Apple executives. The demonstrations are appreciated but Jony Ive is not convinced from a hardware point of view.
- Jony Ive’s point of view. A VR headset is no good because it separates the user from the people and environment around him, and is also clumsy.
- The response of the Technology Development Group. Equip the viewer with an external display that projects the user’s face and gaze to other people. The user wearing the display can see others via one or more cameras.
- 2017 onwards:
- ARKit arrives. Rockwell’s team develops and releases the SDK for making augmented reality apps and games for iPhone and iPad. The software is designed to be easily adaptable to the headset when it arrives.
- Base VS independent device. Rockwell and him want the headset to be attached to a base in order to have more power and better serve the pros, but Jony Ive, again, disagrees. The designer, who in the meantime has already begun to reduce his role in Apple but still has a say in the matter, imagines more a product for the masses to always carry with him – like an iPhone or an Apple Watch, in short. Ultimately, the leadership decides to side with Ive.
- Adaptation. Rockwell and his must then figure out how to adapt the work done so far to the new requirements. The headaches are manifold: with a board you could insert all the hardware functions in a single chip, now they have to be scattered around the viewer – causing problems of latency and communication between one party and another. Meanwhile, even the software, developed with the idea of ​​the base, must be heavily modified.
It seems that, unlike products like the iPhone and Apple Watch, Tim Cook did not participate actively and directed to the project. The initial plan was to launch the headset in 2019, while now it looks like we will have to wait until 2023 – in the most optimistic case the end of 2022. That said, the product should remain more or less conceptually the same – a VR headset with “functionality” AR “thanks to an external screen in which the user’s eyes and face are projected. The source also managed to recover some technical specifications:
- Chip: very similar to the Apple Silicon M2 in the home straight (with the base, the plan was to equip it with a much more powerful M1 Ultra)
- Display: at least 4K resolution per eye
- 14 cameras, some facing outwards, others inwards. In addition to projecting the face, the internal cameras would also be used to create a sort of stylized 3D avatar of the user (Memoji style, so to speak) to be used in virtual meetings and other contexts. Note that the idea of ​​using stylized avatars stems from the limitations of the M2 chip; with the M1 Ultra it was planned to create an ultra-realistic virtual universe, even if there were problems caused by the “uncanny valley” effect (the unpleasant feeling that occurs when the representations of human beings are not perfect and the observer somehow perceives it).
The report finally notes that Apple plans to introduce more conventional-looking AR glasses as well, but it will likely take several more years to see them on the market.
