Apple Intelligence in iOS 18.1: New AI Features You Need to Know

Key Apple Intelligence Features in iOS 18.1

Let’s talk about Apple Intelligence, Apple's new generative AI platform designed to improve user experience across its devices, including the iPhone, iPad, and Mac. Announced during the WWDC 2024 event and further detailed at the iPhone 16 launch, Apple Intelligence aims to integrate advanced AI capabilities into existing applications while, according to Apple, prioritizing user privacy.

The first thing to mention about Apple Intelligence is that it wasn’t available when the new iPhone launched. In fact, iOS 18.0, and the iPhone 16 shipped with no Apple Intelligence features at all. Only iOS 18.1, which I’m running in public beta (and may be publicly available by the time you read this), has some of the new Apple Intelligence features. Apple says it is rolling out its AI features over several upcoming software releases, some stretching into 2025.

As of now - October 2024, there are some interesting Apple Intelligence features available on iOS 18.1, such as:

• Writing tools for rewriting, summarizing, and proofreading text.

• Improved Siri functionality with contextual follow-ups and the ability to type to Siri.

• Cleanup tool in photos for removing unwanted objects.

• Notification summaries.

• Suggested replies in Messages and Mail.

• Custom Memories movies in Photos.

• Article summaries in Safari's Reader Mode.

There are still several highly anticipated features coming in the next update, such as GenMoji (Apple’s AI-generated emojis), Image Playground (to create AI-generated images), and ChatGPT integration.

So let’s evaluate one by one the features that are available today.

Writing Tools

Writing tools are a set of AI-powered tools that are available anytime you are writing text, for example, in Notes. To access the tools, all you need to do is select the text you’ve written, and you’ll notice that writing tools are one of the available options.

Within writing tools, you have some available options.

Proofread. This tool checks for spelling, grammar, and word choice errors, allowing you to review and accept suggested corrections.

Rewrite. This button will launch Apple Intelligence into action and immediately rewrite your selected text. You’ll usually get a more grammatically correct, professional version of your text if you just hit the big rewrite button. However, you also get style options if you want to have it written a certain way. You can choose writing styles like friendly, professional, and concise.

Summarize. You can of course also summarize your selected text into a simple, few sentences, or key points.

Text Reformatting. On top of all that, you can reformat the text to be in list format, or even in a table format. My experience here is that the reformatting to table option will only include text that it deems to be table-worthy. It’s actually an intelligent decision in my books but some people may be surprised by that.

Improved Siri Functionality

Siri also has improved capabilities such as natural language understanding. Siri has a broader understanding based on the upgrades it has received thanks to the new underlying technology of large language models. Even if you stumble over your words, or change your mind midway through your sentence, Siri can still interpret your request. Plus, Siri is context-aware within the timeframe of your open conversation. For example, you can ask follow-up questions without repeating the original context.

An example is asking what the weather is like in San Francisco today, and then asking a follow-up question about the weather tomorrow. Siri will know you’re still talking about San Francisco.

Then there’s Type To Siri, which allows you to have a conversation with Siri without actually having to speak, great for those moments when you are in public or would rather type than talk.

To activate Type To Siri, just double-tap on the very bottom of your screen, on the little home bar. That will bring up a keyboard interface with a message box, along with the familiar Siri glow across the screen. This works just as well speaking, and just the same. Just tell Siri what you want to do or know.

Cleanup Tool In Photos

The Photos app also has a new AI image editing option called the Cleanup tool.  It allows you to remove unwanted elements from photos, including people and things.

The tool intelligently highlights items it thinks you might want to erase. You can see a sort of fluorescent multicolored lighting effect on those items. All you have to do in that case is tap to erase it.

If the item you want to remove is not highlighted, all you have to do is just circle or scribble over it to initiate its removal.

Obviously, your results with this tool will vary depending on how complicated the background is, and how big the object is you’re trying to remove. There are instances where the picture looks considerably worse after editing it because the image it's trying to update just doesn’t work well with the pattern matching. In that case, all the edits you make are reversible so you can either undo each erasure or revert the entire image back to its original form.

However, I’ve found that in many cases the results have been quite good, even convincing.

When you use the cleanup tool, there is a note added in the metadata that states “modified with clean up”, indicating that it’s an AI-edited image.

Notification Summaries

When it comes to one use case that has been a constant with generative AI, it’s the summarization of content. We’ve probably all used ChatGPT for that very purpose. And notification summaries aim to solve a very real problem when it comes to all the information that gets served up on our phones.

Think about it. You probably get notifications for emails, text messages, group chat, and more. Sometimes you can’t tell what the content of an email is just based on the subject line that shows up in your notification. Sometimes you can’t tell what your group chat is talking about by the last message received. And sometimes your partner sends you a message that’s so long you can’t really tell what they’re talking about until you go in and read it.

Notification summaries can handle all of those situations. When you get a notification that has been summarized, you’ll notice a little icon on the notification that looks like a paragraph of text with an arrow wrapping around it.

If an email comes in, instead of a subject line you might get a summary of the email itself. For text messages, you might get a summary of the key points of the message. And for group chats where you’ve received multiple messages, you will get the highlights of what the conversation is about.

The summaries work for other applications as well where you might receive multiple notifications over a short period of time. For example, my security system might send me ten or fifteen notifications in a day due to the system being armed or disarmed, or people who are seen on the cameras. Same with the other smart security cameras I have through the Wyze app.

Summaries are definitely a welcome and useful feature within Apple Intelligence.

More AI Features

And it doesn’t stop there because there are more Apple Intelligence features throughout the operating system. For example, you can now create custom memory movies in Photos. Literally just type in what kind of video you want, based on the photos you know you have - like a memory movie of your child when they were younger and set to a specific type of music you want.

There are also suggested replies in Messages and Mail. For example in messages when someone sends you a text message that seems somewhat open-ended, you will automatically get a couple of suggested replies. They aren’t like paragraphs or anything, more like a two-word reply, but it’s a handy feature.

If you are into reading articles, you will probably already know about Safari’s reader mode. When you go to an article on the web, you can the little icon in the address bar, then tap Show Reader. However, now you’ll see a new Summarize button once in reader mode. Tap it and you get a paragraph or two summary. To me, that’s a nice feature to understand if you want to dive deeper into the article itself, so again a very useful tool.

Conclusion

In conclusion, Apple Intelligence in iOS 18.1 brings several innovative AI-driven features designed to make your device more intuitive and user-friendly. From enhanced writing tools to improved Siri interactions and the Cleanup tool in Photos, these features represent a significant step forward in integrating artificial intelligence into everyday use. While there’s still more to come, such as GenMoji and AI-generated images, the current offerings already add a layer of convenience and productivity that Apple users will appreciate. As AI continues to evolve, it’s exciting to think about how these tools will shape the future of how we interact with our devices.