Apple has once again redefined the boundaries of personal technology with the groundbreaking announcements made at its Worldwide Developers Conference (WWDC) 2024.
The star of the show is undoubtedly Apple Intelligence, a revolutionary personal intelligence system designed to seamlessly integrate into iPhone, iPad, and Mac devices. This innovative system leverages the power of generative AI models combined with personal context to deliver highly relevant and useful intelligence, all while maintaining Apple’s renowned commitment to privacy.
“We’re thrilled to introduce a new chapter in Apple innovation. Apple Intelligence will transform what users can do with our products — and what our products can do for our users,” said Tim Cook, Apple’s CEO. “Our unique approach combines generative AI with a user’s personal context to deliver truly helpful intelligence. And it can access that information in a completely private and secure way to help users do the things that matter most to them. This is AI as only Apple can deliver it, and we can’t wait for users to experience what it can do.”
Enhancing Communication and Creativity
One of the standout features of Apple Intelligence is its ability to enhance written communication. The systemwide Writing Tools in iOS 18, iPadOS 18, and macOS Sequoia allow users to rewrite, proofread, and summarize text across various applications, including Mail, Notes, Pages, and third-party apps.
These tools help users refine their writing, whether it’s for a professional email, a blog post, or class notes.
- Rewrite: Offers different versions of text to suit various tones and audiences, helping users finesse a cover letter or add humor to a party invitation.
- Proofread: Checks grammar, word choice, and sentence structure while suggesting edits, complete with explanations.
- Summarize: Condenses text into digestible formats like paragraphs, bullet points, or tables.
Apple Intelligence also brings new capabilities to image creation with the Image Playground. This feature allows users to create fun and engaging images in seconds, choosing from styles like Animation, Illustration, or Sketch.
Integrated into apps like Messages and Notes, Image Playground makes it easy to enhance communication and self-expression with personalized images.
Taking emoji to an entirely new level, users can create an original Genmoji to express themselves. By simply typing a description, their Genmoji appears, along with additional options.
Users can even create Genmoji of friends and family based on their photos. Just like emoji, Genmoji can be added inline to messages, or shared as a sticker or reaction in a Tapback.
Mail and Notifications: Prioritizing What Matters
Apple Intelligence transforms the Mail app with features designed to help users manage their inbox more effectively:
- Priority Messages: Highlights urgent emails at the top of the inbox, such as same-day dinner invitations or boarding passes.
- Smart Reply: Suggests quick responses and ensures all questions in an email are addressed.
- Email Summaries: Provides summaries of emails without needing to open them, making it easier to scan through long threads.
Notifications are also enhanced with:
- Priority Notifications: Surfaces the most important alerts at the top of the stack.
- Notification Summaries: Helps users quickly scan through long or stacked notifications.
- Reduce Interruptions: A new Focus mode that surfaces only the notifications that might need immediate attention.
Audio and Visual Enhancements
The Notes and Phone apps benefit from the ability to record, transcribe, and summarize audio. This feature is particularly useful for recalling key points from meetings or phone calls.
In the Photos app, new search capabilities allow users to find specific photos and videos using natural language queries. The Clean Up tool can remove distracting objects from photos, and the Memories feature creates personalized storylines from user descriptions, complete with suggested music from Apple Music.
Siri’s Evolution
Siri, Apple’s virtual assistant, has also been upgraded with Apple Intelligence. It now offers richer language-understanding capabilities, making it more natural and contextually relevant.
Users can type to Siri and switch between text and voice, providing a more flexible way to interact with the assistant. Siri’s new design features an elegant glowing light around the screen’s edge when active, enhancing the user experience.
ChatGPT Integration
Apple is integrating ChatGPT access into experiences within iOS 18, iPadOS 18, and macOS Sequoia, allowing users to access its expertise — as well as its image-and document-understanding capabilities — without needing to jump between tools.
Siri can tap into ChatGPT’s expertise when helpful. Users are asked before any questions are sent to ChatGPT, along with any documents or photos, and Siri then presents the answer directly.
Privacy and Security
To be truly helpful, Apple Intelligence relies on understanding deep personal context while also protecting user privacy. A cornerstone of Apple Intelligence is on-device processing, and many of the models that power it run entirely on device.
To run more complex requests that require more processing power, Private Cloud Compute extends the privacy and security of Apple devices into the cloud to unlock even more intelligence.
With Private Cloud Compute, Apple Intelligence can flex and scale its computational capacity and draw on larger, server-based models for more complex requests. These models run on servers powered by Apple silicon, providing a foundation that allows Apple to ensure that data is never retained or exposed.
Independent experts can inspect the code that runs on Apple silicon servers to verify privacy, and Private Cloud Compute cryptographically ensures that iPhone, iPad, and Mac do not talk to a server unless its software has been publicly logged for inspection.
iPadOS 18: Customization and Control
iPadOS 18 introduces more ways for users to express themselves, including new options for customizing the Home Screen, app icons, and Control Center.
Home Screen Customization: Users can place app icons and widgets in any open position, customizing them to appear exactly how they want — light, dark, or with a tint.
Control Center Redesign: The redesigned Control Center offers easier access to frequently used controls, with new levels of customization and flexibility. Users can add controls from supported third-party apps into Control Center.
Calculator with Math Notes: An all-new Calculator app is coming to iPad with iPadOS 18. It includes a Math Notes feature that allows users to type or handwrite mathematical expressions and see them instantly solved in their own handwriting.
Users can assign values to variables when learning concepts or calculating budgets. A new graphing feature lets users write or type an equation and insert a graph with one tap. Math Notes are automatically saved and accessible in the Notes app.
Improved Apple Pencil Integration: Smart Script makes handwritten notes more legible by smoothing and straightening the handwriting in real-time.
Editing handwritten text is as easy as editing typed text, with the ability to add space, scratch out, or paste typed content. Paragraphs will automatically reflow around edits made with Apple Pencil. In Notes, there is a new Image Wand tool in the Pencil palette to create visuals using the Image Playground AI feature.
Gaming and Entertainment
Gaming becomes even more immersive with features like Game Mode and Personalized Spatial Audio, which put players in the middle of the action in upcoming titles like Assassin’s Creed Shadows and RESIDENT EVIL 7 biohazard. The Game Porting Toolkit 2 provides developers with new tools to bring advanced games to iPad, iPhone, and Mac.
Calendar and Reminders Integration
The Calendar app becomes more helpful by showing both events and tasks from Reminders. Users can create, edit, and complete reminders right from Calendar. Reminders get new tools, including the ability to view, create, and edit directly from the Calendar app, access a recently deleted list, and find subtasks in Smart Lists for quick reference.
Apple Maps: Enhanced Navigation
In Apple Maps, users can browse thousands of hikes across national parks in the United States and easily create their own custom walking routes, which they can access offline.
Maps users can also save their favorite national park hikes, custom walking routes, and locations to an all-new Places Library and add personal notes about each spot.
Accessibility Features
Apple Intelligence includes several new accessibility features:
- Eye Tracking: A built-in option for navigating iPad with just eyes.
- Vocal Shortcuts: Enable users to perform tasks by making a custom sound.
Developer Tools and Swift Enhancements
Apple is empowering developers with powerful tools and resources to build incredible apps.
Xcode 16 sets a new standard for developer productivity, offering features and performance enhancements that empower developers to build exceptional apps faster than ever before.
- Swift Assist: Serves as a companion for all coding tasks, seamlessly integrated into Xcode and leveraging the latest software development kits (SDKs) and Swift language features.
- Predictive Code Completion: Uses a unique model specifically trained for Swift and Apple SDKs to suggest the code developers need.
- Dynamic Linking Architecture: Offers a smoother and more productive workflow when switching between previews and code execution.
- Enhanced Testing Environments: Allows developers to see how their apps perform in real-world conditions from the moment they begin building them.
Swift 6 introduces new capabilities aimed at enhancing developer productivity and code clarity, including compile-time data-race safety, which diagnoses concurrent access to memory across a developer’s project at compile time.
Apple is also expanding the availability of Swift to a broader range of developers and programmers, with support for additional code editors that leverage the Language Server Protocol.
An all-new GitHub organization dedicated to Swift will host critical projects for the Swift ecosystem, including the Swift compiler, Foundation, and other key libraries.
The new Swift Testing framework offers developers expressive APIs that make it simple to write tests. It’s cross-platform and easy to learn, so developers can use it to write tests for a variety of platforms and domains. The framework also includes a flexible tagging system to help developers organize their tests and test plans.
Apple Vision Pro and visionOS 2
Apple also unveiled the highly anticipated Apple Vision Pro, a revolutionary spatial computer that seamlessly blends the real and virtual worlds.
Powered by visionOS 2, the Apple Vision Pro delivers a groundbreaking immersive experience, enabling users to interact with digital content in a whole new way.
Key features of visionOS 2 include:
- Hand Gestures: Navigating visionOS is even faster with new gestures that provide quick and easy access to frequently used features.
- Mac Virtual Display: Later this year, Mac Virtual Display will feature a higher resolution and larger size — creating an ultra-wide display that is equivalent to two 4K monitors side by side.
- Mouse Support: visionOS 2 adds mouse support for additional workflow options.
- Physical Keyboard Integration: Vision Pro will now reveal the user’s physical Magic Keyboard — even when they are fully immersed in an Environment or app.
- Home View Customization: Users can now personalize their Home View by rearranging apps and placing them wherever they want, including their compatible iPhone and iPad apps.
- Travel Mode: Adds support for trains, so travelers can experience their favorite apps on the go and even dial in an Environment like Bora Bora to transform their physical surroundings.
- Guest User: A family member or colleague can be added as a Guest User, and their eye and hand data will be saved for 30 days.
Developers and creators will have access to powerful tools to build innovative new experiences for Apple Vision Pro, including the ability to immerse users in virtual environments while they watch videos on popular streaming and entertainment websites using Safari in visionOS 2.
Apple WWDC 2024: Wrapping Up
Apple’s announcements at WWDC 2024 have set the stage for a new era of personal technology, where intelligence, creativity, and immersive experiences converge.
With Apple Intelligence, users can expect a more personalized and efficient way of interacting with their devices, while developers are empowered with cutting-edge tools to create exceptional apps. The introduction of Apple Vision Pro and visionOS 2 further solidifies Apple’s commitment to pushing the boundaries of what’s possible in the realm of spatial computing.