Editing memories, spying on our bodies, normalizing strange glasses: Apple’s new Vision Pro has big ambitions

Apple Vision Pro is a mixed reality headset, which the company hopes will be a “revolutionary space computer that transforms the way people work, collaborate, connect, relive memories, and enjoy entertainment,” which will begin shipping to the public (in the United States) later this week.

Critics have doubted the appeal of the face-worn computer, which “seamlessly blends digital content with the physical world,” but Apple has previously sold up to 180,000 of the devices valued at $3,500.

What does Apple think people will do with these expensive peripherals? While uses will evolve, Apple is focusing its attention on watching TV and movies, editing and reliving “memories” and, perhaps most important to the product’s success, making its customers not look like weirdos.

Apple hopes the new device will redefine personal computing, as the iPhone did 16 years ago and the Macintosh did 40 years ago. But if successful, it will also redefine privacy concerns as it captures enormous amounts of data about users and their environments, creating an unprecedented type of “security.”biospatial surveillance“.

Spatial computing

Apple is careful with its branding and how it packages and describes its products. In an extensive set of rules for developers, the company insists that the new headphones should not be referred to as “headphones.” What’s more, Apple Vision Pro does not use “augmented reality (AR), virtual reality (VR), extended reality (XR) or mixed reality (MR)”: it is a gateway to “spatial computing.”

Spatial computing, as described in 2003 Ph.D. thesis by American software engineer Simon Greenwold, is: “human interaction with a machine in which the machine retains and manipulates referents to real objects and spaces.” In other words, the computer can interact with things in the user’s physical environment in real time to provide new types of experiences.

Vision Pro has big shoes to fill for new user experiences. The iPhone’s initial “killer apps” were clear: Internet in your pocket (including portable access to Google Maps), all your music on a touch screen and “visual voicemail“.

Sixteen years later, all three seem ordinary. Apple has sold billions of iPhones and some 80% of humans Now he uses a smartphone. Its success has all but wiped out earlier tools like paper maps and music CDs (and the ubiquity of text, picture, and video messaging has largely eliminated voicemail itself).

killer apps

We don’t yet know what the killer applications of spatial computing might be (if any), but this is where Apple is directing our attention.

The first is entertainment: Vision Pro promises “the definitive personal theater“.

The second is an attempt to solve the social problem of walking around with strange headphones that cover half your face. An external display on the glasses shows a constantly updated representation of your eyes to Offer important social signals about your gaze. to those around you. Admittedly, this seems strange. But Apple hopes it will be less awkward and more useful than trying to interact with humans wearing blank aluminum ski goggles.

The third is the ability to capture and relive “memories” – recording and playing back 3D images and audio of real events. Critics have found it surprising: “this was things from my own life, my own memories. She was reproducing experiences she had already lived.”

Apple has proprietary tools to select, store and annotate digital “memories.” These memories are files, and potentially products, that will be shared in “space videos.” recorded on the latest iPhones.

Biospatial surveillance

There is already a huge infrastructure dedicated to helping tech companies track our behavior to sell us things. Recent investigation Facebook, for example, receives data from an average of about 2,300 companies about each individual user.

Spatial computing offers a radical change in this tracking. To work, spatial computing records and uses large amounts of intimate data about our bodies and our environment.

One study on headphone design They observed no less than 64 different streams of biometric and physiological data, from eye tracking and pupil response to subtle changes in the body’s electromagnetic field.

Your face tomorrow

This is not “consumer” data like what brand of toothpaste you buy. It is more similar to medical data.

For example, analyze a person’s unconscious movements It can reveal your emotional state or even predict neurodegenerative diseases. Is called “biometrically inferred data“As users are not aware that their bodies are abandoning them.

Apple suggests that it won’t share this type of data with anyone, and Apple has proven to be better than most companies when it comes to privacy. But biospatial surveillance uses more of ourselves for spatial computing, in ways that are expanding.

It starts simple enough in the pre-order process, where you need to scan your facial features with your iPhone (to ensure a perfect fit). But that’s not the end.

apple patent on memories It’s also about how to “guide and direct a user with attention, memory and cognition” through feedback loops that monitor “facial recognition, eye tracking, user mood detection, user emotion detection.” user, voice detection, etc. [from a] Biosensor to track biometric characteristics, such as health and activity metrics. […] and other health-related information.”

Social questions

Biospatial surveillance is also the key to Apple’s attempt to solve the social problems created by wearing headphones in public. The external display that shows a simulated approximation of the user’s gaze is based on constant measurement of the user’s expression and eye movement with multiple sensors.

Your face is constantly mapped so that others can see it, or rather see Apple’s vision of it. Likewise, as passersby approach the Apple Vision Pro sensors, Apple’s vision of them automatically translates into their experience, whether they like it or not.

Apple’s new vision for us (and those around us) shows how the requirements and benefits of spatial computing will raise new concerns about privacy and social issues. Extensive biospatial surveillance that captures intimate biometric and environmental data redefines what personal data and social interactions are possible to exploit.

This article is republished from The conversation under a Creative Commons license. Read the Original article.