Apple planning to add Vision Pro-like gestures to upcoming iPhones, iPads, and Macs

Apple is reportedly planning to expand the innovative gesture controls introduced with its Vision Pro headset to other devices, such as iPhones, iPads, and Macs.

The Vision Pro introduced a series of intuitive gestures for interacting with apps, like moving windows and resizing documents, making these actions feel natural and seamless. Now, Apple seems eager to apply these gestures more broadly across its product line.

The recent patent application, titled “Devices, Methods, And Graphical User Interfaces For Using A Cursor To Interact With Three-Dimensional Environments,” hints at Apple’s ambition.

This filing suggests that Apple envisions a future where these gesture controls could be used with various devices. While Apple has explored similar ideas before, including a 2009 patent, the success of the Vision Pro gestures appears to have provided renewed momentum.

The patent includes a variety of illustrative drawings and diagrams, many of which focus on the Vision Pro. These include component breakdowns of the headset, hinting at how it detects user gestures and transmits these signals to a Mac or another device.

Notably, the drawings also depict users interacting with an iPad or a traditional desktop computer, neither of which involves wearing a headset. This indicates that Apple is considering how to implement gesture controls on devices without relying on the Vision Pro hardware.

Apple’s patent application describes these devices as providing “computer-generated experiences,” encompassing virtual reality and mixed reality via a display. This suggests that gesture controls could be integrated into existing devices equipped with Face ID sensors, which already scan users’ faces for authentication. It’s not a huge leap to imagine these sensors detecting hand movements to control device functions.

Currently, Face ID sensors are used for more than just unlocking devices. In iOS 18, these sensors allow users to navigate their iPhones by simply looking at them. The new patent application focuses on how to accurately recognize gestures and dismiss false positives, such as someone scratching their nose.

The system is designed to determine whether a hand movement meets specific criteria to trigger an action, like moving a cursor or adjusting a window’s position. If the criteria aren’t met, the system ignores the gesture.

While the patent mainly describes the technical process of gesture recognition and the resulting actions, it hints at a broader vision for Apple’s devices. The company has long resisted adding touchscreens to Macs, citing ergonomic concerns about users reaching for the screen. By implementing gesture controls, Apple could offer an alternative that maintains user comfort while enhancing device interaction.

This patent application, credited to inventors including Evgenii Krivoruchko, underscores Apple’s ongoing commitment to enhancing user experiences through innovative technology. Krivoruchko has several patents related to using user attention to control elements in 3D environments.

As Apple continues to push the boundaries of device interaction, users may soon find themselves waving their hands to command their iPhones, iPads, and Macs, making touch-free control a reality.