Back to the articles

iOS 27 and Apple Intelligence : How Apple is Reinventing Accessibility with AI

20/5/26

When AI becomes about inclusion

Apple chose the eve of Global Accessibility Awareness Day to unveil its new features related to iOS 27.

The timing is no coincidence; every year, Apple uses this event to showcase its advancements in accessibility, but this time, the change is fundamental.

For the first time, Apple Intelligence, its embedded AI infrastructure, is at the heart of almost all announcements. It's no longer a cosmetic addition; it's a fundamental shift in how people with disabilities can interact with their devices.

The features presented apply to iPhone, iPad, Mac, Apple TV, and Apple Vision Pro. They will officially arrive with iOS 27, the first public beta of which will likely be launched at WWDC (Worldwide Developers Conference) on June 8.

VoiceOver and Magnifier: AI as a Pair of Eyes

For blind or low-vision individuals, Apple has significantly improved VoiceOver and the Magnifier app.

VoiceOver is Apple's long-standing tool that reads aloud what's on the screen. With iOS 27, it gains precision thanks to Apple Intelligence, capable of generating much more detailed descriptions of images throughout the system, including photos, scanned invoices, and personal documents.

The Magnifier app, meanwhile, takes another step forward: you can now point the app at an invoice and ask for the total amount, which will then be dictated. The app can also be fully controlled by voice, to zoom in or activate the flashlight, without ever touching the screen. This type of interaction, which seemed like science fiction just two years ago, is becoming a concrete reality for millions of people.

The iPhone's Action button can also be configured to ask real-time questions about what the camera sees, with support for natural language queries.

This is particularly useful for visually impaired individuals who struggle to find an object in a room; pointing the camera and asking the question aloud becomes a direct answer to this daily need.

Automatic Captions for All Videos

One of the most notable announcements concerns deaf or hard-of-hearing individuals: iOS 27 will allow for the generation of automatic captions for all videos that don't have them, whether they are personal clips, videos received from loved ones, or online content.

What distinguishes this implementation is that processing occurs entirely on the device. Caption generation happens in real-time and locally, ensuring the privacy of communications. The feature will be available on iPhone, iPad, Mac, Apple TV, and Apple Vision Pro, although it will initially roll out in English in the United States and Canada.

The Accessibility Reader has also been enhanced to handle complex documents like multi-column scientific articles, with on-demand summaries and integrated translation that preserves the user's formatting preferences.

Vision Pro Reimagined for People with Limited Mobility

Apple's mixed reality headset includes two particularly significant new features for people with limited mobility.

The first is spectacular: for Vision Pro users who cannot use a joystick, Apple uses the headset's eye-tracking system to allow users to control an electric wheelchair with their eyes. The feature will initially be compatible with Tolt and LUCI systems in the United States, with efforts underway to expand compatibility to other manufacturers.

The Vision Pro will also support facial gestures for taps and system actions, as well as a new way to select items with the eyes when using Dwell Control.

For individuals who cannot physically interact with a screen, these two combined functions represent a substantial advancement.

Conclusion

What's striking about this wave of announcements is that Apple's accessibility features have always ended up reaching far beyond their initial target audience.

Automatic captions will be useful to anyone watching a video in a noisy environment or without headphones, and spoken image descriptions will benefit both visually impaired individuals and users who are driving and want to understand a document without looking at their screen.

All these processes are performed locally on devices, which Apple explicitly emphasizes. This choice ensures data privacy without relying on remote servers. However, recent hardware will be required to benefit from these features, specifically an iPhone 15 Pro or later, or a Mac equipped with an Apple Silicon chip.

The upcoming WWDC (Worldwide Developers Conference) on June 8th is expected to reveal the full scope of Apple Intelligence's ambitions for this generation of systems.

What the brand has just announced regarding accessibility is likely a preview of what awaits all users this fall, whether they have specific needs or not.

Frequently asked questions

When will the new iOS 27 accessibility features be available ?
Drowpdown Klark
Do these features require a recent iPhone ?
Drowpdown Klark
Do iOS 27 automatic captions work with all videos ?
Drowpdown Klark
How can Vision Pro control an electric wheelchair ?
Drowpdown Klark
⚡Early access: The platform is in development mode.