Apple's AI advantage... and challenge

I've mentioned before how I think what's going on behind the scenes with Artificial Intelligence, beyond the assistant- and chat-based interfaces, is every bit as exciting as what we're already seeing. Apple's kept quiet on their own plans in the area for a long, long time. This week, though, Tim Cook let just a little bit out.

Talking with Nikkei, Cook said:

Apple intends to capitalize on AI in various ways, in cooperation with Japanese companies. AI is "horizontal in nature, running across all products" and is used "in ways that most people don't even think about.""We want the AI to increase your battery life" and recommend music to Apple Music subscribers, he continued. As another example, he said AI could "help you remember where you parked your car."

It's the "increase your battery life" part that's especially interesting here. There's no way to know what Cook is referring to specifically, but some reasonable extrapolation that can be made, given Apple's products and services to date.

Apple holds a massive and growing advantage in silicon — they make their own chipsets from the A10 Fusion that drives the iPhone and, soon, the iPad, to the M10 motion hub, the S2 system-in-a-package for Apple Watch, the W1 wireless chip in AirPods, the timing controller for the 5K iMac, and the list is growing.

That means they can build the atoms specifically to support the bits, and vice-versa. It's why iPhone can perform better with fewer cores, and have greater power efficiency even with a smaller battery.

We've seen Apple's AI, Machine Learning (ML), and Computer Vision (CV) efforts to date in everything from Siri to Proactive to Photos search. That's all in the software. What happens when it's baked into the silicon?

Previously we could launch apps with icons. Now we can launch actions with 3D Touch. What happens when iPhone has a reasonable chance of predicting what app and what action you're going to launch next — at the chipset level?

What happens when it can intelligently manage power not just based on what you are doing, but what it's learned you will be doing?

Worst case, if it messes up or you do something extraordinary, it's a wash. Best case, the order of efficiency is improved significantly.

Cook is remarkably straightforward when he speaks. When he says AI is "horizontal in nature, running across all products," take him at his word and consider the implication, then consider it again from a company that has full vertical integration from server to interface to device chipset, and will eventually have AI running horizontally across all of it.

A10 Fusion and Siri wouldn't have nothing on that stack.

And best of all, it fits in perfectly with Apple's stance on security and privacy. They still won't need to harvest your data to feed their machine. They can happily learn your data locally, combine what's needed from the public cloud, and give you all the benefits.

It's an incredible advantage, if Apple can take advantage of it. Because that's the equally big challenge here, and it's something the company has struggled with since ramping up its AI efforts with Siri.

Google, Amazon, and Facebook don't go down to the metal, at least not yet. But Apple needs to not only reach for those clouds — they need to nail them.

Rene Ritchie

Rene Ritchie is one of the most respected Apple analysts in the business, reaching a combined audience of over 40 million readers a month. His YouTube channel, Vector, has over 90 thousand subscribers and 14 million views and his podcasts, including Debug, have been downloaded over 20 million times. He also regularly co-hosts MacBreak Weekly for the TWiT network and co-hosted CES Live! and Talk Mobile. Based in Montreal, Rene is a former director of product marketing, web developer, and graphic designer. He's authored several books and appeared on numerous television and radio segments to discuss Apple and the technology industry. When not working, he likes to cook, grapple, and spend time with his friends and family.

  • Siri
  • Rene, your thoughts (I would characterize as probable insights) are a primary reason I continue to follow iMore. Thank you for your contributions! [Serenity Caldwell's articles (with associated opinions supported by data she has gathered via trying/testing) also falls in this category.] Sent from the iMore App
  • I agree that they make the best ARM processors in the business. They are a yr or two ahead of the rest. Problem is Google is probably 3 to 5 years a head of Apple in AI. Googles AI doesn't rely on the processor in the phone. If it did then you'd be correct that the ARM advantage would be a big benefit. Also if you look at something like Google Home it's a benefit that it's not relying on the ARM processor as if it did the cost of all these devices would be very expensive. Think of your Apple Watch 2. It's not powerful. All the Android phones are using silicon that is much faster than one in the watch. If Apple AI needs a wearable such as a watch to need the power of A10 Fusion then Apple is in big trouble. If the wearables have to connect to another device then not only do you need that device but it might not work. Think how the watch needed the phone to run apps with Watch OS 1. It was slow and unreliable. Google is in the driver seat here. Apple will have to pull a rabbit out of their hat which is possible but not likely.
  • I agree with this comment. I just recently switched to iPhone from Android (I'm a perpetual
    phone switcher). Apple is so far behind Google in the AI arena that they may never catch up. Google Now works great especially for travel. I don't think that advanced ARM chips in the iPhone can close the gap with the server farms Google has at it's disposal.
  • Apple is way behind both Google and Microsoft. Amazon is a sleeper, as well. Apple is behind both technologically and the in UX. Because of their on device philosophy, the UX across devices is extremely poor compared to Google and Microsoft. I am no longer sure it's worth the "privacy," you gain, either. As far as I'm concerned, it's basically conditioned me to not care to use the features, because it's double the work to keep all of your Apple devices in lock step compared to Windows or Google devices. Your Xbox, Surface, Laptop, Desktop, etc. will just pull down all of your stuff and all work fairly identically as far as these features are concerned. Same with Google Home, Google Watch. Android Phones. Android TV. Chrome OS... Not so with Apple devices. Good Luck synching those faces across devices! :-P Honestly I'm thinking about going to the Google Pixel XL largely for this reason.
  • Apple just sacked it's AI team and hired a new AI head from the outside, who is bringing in his own people. So much for any advantage.
  • I've read about Salakhutdinov. Can you cite a reference about the sacking of the "old" AI team?
  • More of an educated guess as I never considered your take; that Apple had no head of AI nor an existing AI team I'm assuming that there was an old head of AI and that team was based in Cupertino. A bit of a long commute from Carnegie Mellon for anyone on the new team. Perhaps that assumption is wrong and Apple just had general engineers on the project.
  • That's cool and all. I just wished they'd open up the AI to everyone. As of now, Apple only uses Siri for apps and services they don't provide. So, I can't use a different map app or music app or whatever with it. Both Google and Microsoft are winning in those areas..
  • Siri will launch any app. And in iOS 10, there's no reason that you couldn't say, "Find 123 Main Street with Google Maps."
  • I think you're missing the point. Yes, you can open apps. That's been around for as long as I can remember. But, I can't get the apps to do anything else like I can using stock apps. For example, I cannot tell Siri to take me home using HERE WEGO which has offline maps, therefore not using my DATA. I can't tell Google Play Music to create a playlist based on Coldplay. I can't do that on the stock Music app either unless I'm subscribed to Apple Music, Something I could do before iOS 10, for obvious reasons...$$$
  • iOS 10 allows Siri to interact with apps, so long as the app supports it. For example, you can say "Get me an Uber taxi" if you have the Uber app installed and it will give you the option to book the taxi straight from Siri
  • Hence my original comment..."As of now, Apple only uses Siri for apps and services they don't provide." So, they'll allow apps that don't directly compete with them.
  • I've never seen any documentation on this, as far as I know Apple allows Siri for any apps in iOS 10. If you can find the proof of this then let me know, that seems wrong to me
  • using "hey siri" for two weeks and does not work hands free at all. all the time asking me to unlock my iphone so what is the point to have it? when i have to unlock my phone anyways maybe i do something wrong? i know for sure it just works one way ether privacy or AI. because SIRI need the information from somewhere
  • What Siri commands are you trying to do?
  • Apple opens a R&D center in Yokohama, Japan, that will focus on "component technologies", "deep engineering". Apple "intends to capitalize on AI in various ways, in cooperation with Japanese companies." Maybe they are in search of a supercomputer, a Fujitsu perhaps, to buy or rent, to be used in the creation of these "component technologies" and "deep engineering", who knows? Whatever the reason to open this R&D facility in Japan is, connecting the dots in that interview leads us to the conclusion that Apple's AI efforts may have a specific hardware dimension.
  • AI is trendy, thanks for your writing.
  • Looks like Apple fanboy article trying to stay relevant with Google, Microsoft and other companies superior AI efforts.
  • Looks like an article which is apparently attracting Android fanboys
  • He is right though. Apple's work with AI sucks compared to their competition. Posted from my Nexus 6P
  • iOS's AI has been inconsistent at best. With iOS 8 & 9, when I connected the aux cable in my car the screen would light up with playback controls of whatever I was last listening to, even if the app were closed. Now in iOS 10, it doesn't even if the app is the front most. I might get the podcast app in the bottom left, but sometimes it's Words With Friends.
    iOS 9, I could tell Siri, "remind me about this email in 30 minutes." It worked well. Now in iOS 10, 90% of the time it creates a reminder with no link simply titled literally "this email". WTF?!
    I still can't voice dial saved Company contacts. For example, I have several of my distributors & a couple tech support numbers saved in my contacts. I ask Siri to "call Crestron tech support" (Crestron is the company name, I made a custom phone label as tech support, like mobile/work/etc). It replies, "Sorry, I don't see Chris Tron in your contacts" WTF?!? So I ended up adding the f'n name Chris Tron to that contact!
    Voice recognition is only about 75% accurate across the board. I can't whisper to my phone or watch.
    Using Bing for internet searches is ridiculous.
    OH... and my favorite - I enter my service calls in my calendar WITH an address. I finally get travel reminders, however, I ask Siri, "give me directions to my next appointment" and all it does is tell time the title of the appointment.
    Asking Siri to "read the last message from [my wife]" finally usually works correctly in iOS 10. It used to read me messages from other people from hours ago.
    Point is- I try to use Siri and any other "intelligence" iOS has to offer, but it falls flat 15-25% of the time. Imagine if a certain feature of your car failed 15-25% of the time, like the brakes. Apples & Oranges, maybe, so let's say the power windows had to think about it, think about it, think about it... sorry, Power Windows (::Siri) is unavailable at the moment, please try again later. Sent from the iMore App
  • Bing is fine. Crying about Bing is like a consumer demanding photoshop to edit JPEGs when they're given PaintShop Pro. It sounds petty. I've tested both Bing and Google, and 9.9x/10 they give basically the same results. It's part of the reason why I even use Bing in Safari, for consistency, as well as in my iMac's browser. I don't knock you for preferring Google. I think their search result pages look better than Bing's (though I prefer Bing's image search). However, there's nothing bad or ridiculous about using Bing search, at all. People in other countries may have a different take, maybe. People on the internet always overstate and exaggerate. I'm speaking from a US perspective. To get back to the main point of your post... Siri is awful compared to Google Now/Assistant and Cortana. Apple's policies, as a result of their rhetoric and attempt to capitalize on privacy awareness has cuffed them. Unless they've been doing even more groundbreaking research than Microsoft and Google have been doing in this area (and that's probably close to impossible), they are going to be laggards for quite a while. The biggest thing about Siri was never superior AI. She always leaned upon Google or Microsoft, WolframAlpha, and other sources to source data. The big thing was the user interface and user experience she ushered in when she was released. It was something new to many people, therefore it was exciting. But she was never great and she only looked as good as she did because other companies simply were not investing in developing user interfaces to their Data Graphs. Now that they are, we are seeing the full effects of the disparity between companies like Google and Microsoft, and those like Apple and Samsung in this area. It isn't even a race at this point, because both MS and Google are too far ahead. Also, Google and Microsoft both own the platforms and services umbrella they need to be self-contained in this, and are dominant in enough of their key areas to make it very hard for competitors to pass them. For example, Apple's speech recognition for no-English languages is weaker than Google and Microsoft, and it's probably directly related to their high device prices. People in European and developing nations cannot afford those products, so they don't get as much data points to improve those systems. Microsoft and Google don't have this issue with Windows and Android. Also, they both own Translation services, which are also huge in developing these systems (as it gives you tons of idiomatic/colloquial speech data points, which is why Cortana and Google Now are much better than Siri at dealing with normal speed, fluent, idiomatic speech with a wide range of accents).
  • Quote: "What happens when it's baked into the silicon?" What's baked into the Silicon is software. It's all basically software. Code, algorithms, etc. I think that's the fundamental thing you don't seem to understand. It doesn't matter how well your chips benchmark, if your software is weaker, you end up with phones like the Pixel shooting better pictures using QC Chips and ISPs based largely on Google's camera software. You end up with Google Now and Cortana (and Alexa) embarrassing Siri and leaving you with no answer to them. You end up late to the "Smart Photo Stuffs" game (behind both Google and Microsoft). Apple is in a bad position for this, because AI is all about mapping behavior. Microsoft and Google are both set up MUCH better than Apple for this. They have a better services umbrella. They have a better strategy for delivering the benefits to their users, and they're willing to deal with the compromises this entails. They have years and years (Decades, for MS) of in-house R&D put into these areas. They didn't just buy a few cheap startups and decide to integrate some features. They have been taking this serious for a long time. The only thing they didn't do, until relatively recently, is develop the user interfaces to allow consumers (and businesses, in some respects) to take advantage of this. This is why Siri was great. It was a new way to use a phone. It was never that good, but it was the best option given the lack of attention to this area companies with better tech were paying attention to it. It took literally no time for products like Google Now, Cortana, and Alexa to leapfrog Siri. They literally did it the day they were released. Apple has had years to get Siri to that level. She isn't there because they don't have the capability to move much faster, to be frank. Also, they have painted themselves into a corner technically given the way they've chosen to implement many of their AI-related functions. A10 Fusion doesn't mean anything. It's just another CPU. This is like saying Photoshop will instantly be better simply because it's running on an Intel CPU instead of an AMD CPU. It's faster, yet, but many of the advantages that Apple touts about their CPUs are routinely trumped by software and services from other companies simply because they're better at software and services than Apple.
  • You should be the editorial director for iMore (MN). At least you have rational thought. Posted via the iMore App for Android
  • I agree! Sent from the iMore App
  • As do the other people who work at iMore
  • There is something on your nose Danny...and its brown...and it smells funny.....what is it?
  • It's probably you because you're full of sh*t
  • The more I read about it, the more I love Apple philosophy
  • Apple is a big company, much bigger than Microsoft and Google. Big companies move slower in adopting a novelty or changing technology direction. Smaller companies may move faster in such avant-garde endeavors but they cannot reshape the technology landscape at the end. Apple can. Apple's target is not to be the first, but to be the best. iPhone is not the first, but it is the best, otherwise smartphones exist since the turn of the century. Google does not care whether its AI efforts will stick to the wall or not. It doesn't care even for Android. After Oracle's Java VM lawsuit it fired Android's creator and demoted Android by subordinating it to the Chrome team. Android is just a stolen and foster child in Google's technology sandbox. That sandbox will not produce an AI product much valuable than Android.
  • Wow! I hate to call someone dumb but this point of view sure is. Big companies miss the revolution all the time. Do You need a history lesson? Google ( 3 or 4 man founding group) killed everyone on search on the internet. A 4 man founding group built a company that is now bigger than Microsoft. Yes bigger. This is in the field the MS should have owned! You don't know what your talking about. IBM use to be Apple. Now where is it. It's way down the list. Google is bigger than IBM. The size of a company only allows you to scale something. If your tech isn't as good as anothers your size doesn't help especially in software since software can be run anywhere. You can spin up servers from many companies and you don't need money to build them. It's the easiest time to take a superior technology and beat a big company. Let me give you resources to learn about this. There are a lot of podcasts that will help you why your wrong.
  • Apple is definitely barely not bigger than Microsoft. And manpower means nothing if competitors have greater talent. Look into Microsoft Research and Google's equivalent.
  • Just because a company may have more cash on hand doesn't mean it's really bigger. Just to show the power of MS I use this as an example: Just shut the doors of MS, think of the world-wide effect it would have on tech products. PC's, Tablets, Servers, & other devices. The biggest effect of course comes from an Operating System & Software perspective. Apple on the other hand, if they closed their doors today, what type of world-wide effect would it have in comparison?
    Phones definitely be effected world-wide.
    Would PC's be effected? Nope.
    Would tablets, yes.
    Servers? Nope. AI needs tons of data. Both MS & Google have tons of it &, has been mentioned, they have invested a lot thru the years. They are currently applying it & people are using it. AI needs powerful hardware on the server side as well. MS has servers that can handle this type of service. MS even has Azure. MS has also been creating Apps for other platforms, like Andriod or iOS. I know Cortana doesn't work that great on iOS, but it's because they don't access to the core of the OS like they do on Windows. People are going to want AI on their PC, tablet, phone, & many other devices. As has been mentioned about big companies missing revolutions or completely fumbling, MS has definitely done that many times. Their phone practically doesn't exist in comparison to Apple or Google...That's called completely missing the boat & not taking it seriously. I'm a long time Windows user (Even their phone), I now use (Which I previously did use) the iPhone & iPad. I love iOS. I have been trying to use my Windows 10 on a Surface Pro 3 less & less compared to my iPad Pro.
    I can not state to others that MS is a small company, it has more influence in the broad sense than either Google or Apple. It will take time for that to change & honestly, it has been in the process of losing that grip already. But they are getting more innovative & it's working. A truly powerful AI would allow you to use it on multiple platforms & multiple devices...Phones, PC's, stand alone iOT devices, appliances, & anything else. Amazon has a big footprint also. Google is doing quite well, but I was just comparing Apple and MS for now. I'm a firm believer that Apple can develop a bigger, broader reach into the AI world. I personally do believe they have "more up their sleeve" than what we think concerning AI. Apple may be looking at approaching AI from a different angle, which is common for Apple to do & be successful at it. I know they are working with IBM some & they have powerful hardware & software. I sometimes wonder what it would be like if Steve Jobs was still running Apple & what it would be like now. I'm not against Tim Cook, but Jobs made Apple fantastic. I personally can live without more advanced AI. I know the direction of society is completely moving towards such advanced technology that they will practically be completely dependent on it. That's a scary thought. People won't know how to do even basic tasks anymore.
  • If Google has really a working AI product the best thing it can do with that is to sell it to Apple. In fact it is in better position with Apple than a few years ago: it couldn't sell Maps to Apple because Maps was already a bad product (satellite imagery decade old at best) and there was the tension raised by Android (remember Steve Jobs would go nuclear against Android). Now that Google has demoted Android it may be in a better position to make a deal with Apple. Otherwise Apple will eventually do that AI thing and will do it better as always...
  • Nothing you said makes sense.
  • Another one where I read it and say wow! This person has no idea what they are talking about. Top 10 downloaded apps on iOS as of may 2016 The Full List in order.
    Facebook Messenger
    WhatsApp Messenger
    Find My iPhone
    Google Maps
    iTunes U I seem to see 2 google apps in the top 10. Youtube and Google maps at number 8. Google Maps is still a highly used app on iOS.
  • "We've seen Apple's AI, Machine Learning (ML), and Computer Vision (CV) efforts to date in everything from Siri to Proactive to Photos search. That's all in the software. What happens when it's baked into the silicon?" It's still in the software, duh. The above is like saying that the software which runs the Qualcomm modems or Cirrus Logic soundchip or (insert a list of a dozen or so Apple's hardware suppliers here) is not software since it is embedded in a particular chipset in lieu of another chipset. In short, one cannot implement an algorithm without software, since algorithms are software, and vice versa. Where the software happens to reside is quite irrelevant to the matter. Computer hardware without software cannot compute.
  • Also, on a language like Finnish, which is inherently difficult for AI, Siri does quite poorly compared to Google Now. The latter seems to be years ahead not only in tangible language recognition prowess but also in real world usability, since the Siri command set is very limited even still in iOS 10. One wagers the underlying technology is simply more robust on the Google product, allowing their AI team to extend its capabilities effortlessly beyond the major languages and initial usage scenarios to entirely new, nascent areas. It was, one thinks, built from the ground up to be more universally applicable than Siri. So Apple's AI is, in fact, a limiting factor for them, at least against companies like MS and Google that have done their own R&D instead of simply buying their way cheap and quick into the technology.
  • It's because Google and Microsoft have invested more and used their products as a vector to collect data to J.prove the products. People complain about data collection, but this is the result. Better products. These are areas where results speak more than market, and those results are a good justification for those policies.
  • Regarding "tons of data. Both MS & Google [supposedly] have", that data does not belong to them. What they have is tons of "metadata", not the whole web for god's sake :-))) The data required for the AI resides on universities, mostly government, research institutions and alike... And reaching that data to feed any AI system is just a contractual issue. And that data only is valuable for any AI system, not everyone's crappy web history...
  • Just a thought or two or three...A search engine alone crawls the web no matter if you go to that site collecting data. Software can read that webpage and know how it's designed without you looking at it. Now how much data they are able to collect from each page may vary by design or legality. Of course, they access every site either.
    Just a thought about search:
    Now, if you search for a product to purchase, the data amassed by it can be the terms you used, what links you clicked on, where on the page the link was located, the final site you purchased from, method of payment, how long you were on that site, what other sites did you visit that day. What style of the webpages they were that you visited, including the colors, objects, fonts, etc were in that page. All that data can be "priceless" Some data MS or Google may not get purely by searches, but with a deep desire to obtain data and deep pockets, they can buy some of that data from schools, businesses, governments, even basic surveys. Or just by working with partner companies. How you use your OS and the apps may be recorded as anonymous and that's fine. If you choose to right click on a file and select 'rename' or you'd rather simply click on the rename button is all part of this massive data collected. MS does use even that data for design changes for Windows...Lots of data is available. One of most important data needed for AI for devices/software interacting with human is learning human behavior. Scientists even go to Facebook to study human behavior. There is plenty of data available. Whether you collect it yourself or buy it, it's out there. Even Google has gotten into trouble for scanning university students emails for 'commercial purposes' is honestly irrelevant in today's world. When you have the most widely used mobile OS (Google) or you have the OS on around 90% of the worlds PC's (MS) you have access to an immeasurable wealth of data. When you have "digital assistants" that are already learning human behavior, you are collecting data. Anonymous data is still data. Knowing a definition of a word is data, but knowing context of its usage is data that is freely (or at a cost) available. Yes, look at some of those user agreements for the free services you're getting from MS and Google and I know I'm giving some of my data away...That's how I pay for that free service. Now, granted, not all the services collect
    that data. Don't limit data collection to purely one company's search engine. If you want MS to custom design software for your hospital, you may need to give them basic data sets so they know how to make it. That's just basic data that they now have. What matters is if you are designing software to deal with and utilize that data, even if you have gotten live data yet...because it will come. Do they have tons of data? Yes. Does Apple have that much data in their caverns? I surely don't know, but I'm betting they could buy a lot. Hardware, software, and raw data must marry up to make it all work. Sent from the iMore App
  • I meant, of course they CAN'T access every site - That's a given Sent from the iMore App
  • Apple cannot buy Google and Microsoft data sizes. They also aren't set up to collect their own so they'll hemorrhage money on this, while their Competitors have up to the moment, accurate, self Contained data collection machines. This is why Siri is so bad Compared to Corral and Google. Their competitors just eat the bad less because they know how important it is to get this information, for building and improving systems. Apple tried to white knight the privacy issue, and is behind the 8 ball for every useful feature dependent on it as a result. Ots a big deal. I'll likely get the pixel because one can be sure that Apple will take forever catching up to what those companies are offering (and they definitely aren't standing still either). Siri is not on par with Cortana on the desktop or mobile. It isn't on par with Google on mobile. Their services are laggard, and services like iMessage and FaceTime simply aren't looking like great justifications these days. Everytime they talk about privacy it is sounding like an excuse for laggard momentum these days. There are things about the pixel which aren't perfect, but I'm jncreasi fly feeling like its an liberally better balance of benefits and Compromises than the iPhone.
  • I am moved to comment because this is the 2nd time in a month I am stunned by the gushing sycophancy of the writer of this article. We all like and enjoy Apple products; that is why we come to websites offering insight and review. A few weeks ago, the author shared his delight with the $150 AirPods by saying that sound quality seemed at least as good as the regular cheapo Apple wired EarPods. Well that's a relief, but why dwell on sound quality? It's only a headphone after all. So, after reading the above, I felt moved to comment because yet again because the above article borders on the absurd and is just asinine. Apple's SIRI is so far behind that it can not even tell me the mobile number of my wife, or anyone, in my contacts. Try it yourself - Ask for the mobile number of anyone and you'll get "Sorry, there's something wrong. Can you please try that again" So, SIRI isn't capable enough to give me a phone number on my not so smart phone. What hardware changes would render it so capable? So, let's be clear, Apple's AI is generations behind Google's which is entirely software based residing in the cloud. There is nothing inhibiting Google's vastly superior assistant and AI migrating beyond Pixel into every other Google touched device, other than a marketing decision. Google's AI has little to do with the device and everything to do with the time, emphasis, data mining and software engineering that has gone into its development. I value Apple's treatment of privacy but Apple has neglected SIRI and its AI is years behind.
  • When it comes to AI, Apple can not win against Google, it's as simple as that. Sent from the iMore App
  • Well, they could in the future, who knows what will happen? Google is probably winning now though, that doesn't mean they'll have that title forever. Technology changes fast