Siri 'crushing' competitors at language accuracy

By Rene Ritchie
last updated

Siri, Apple's virtual personal assistant, has been improving steadily since its introducing in 2011. Streaming text to speech is one of the most recent and obvious areas of improvement, but Siri has also been adding new features and better language support. In a recent test that spanned languages, Siri scored a remarkable 76% for accuracy. That's compared to 46% for Google Now and 42% for Microsoft Cortana. Sophie Salonga, writing for Venture Beat:

In conclusion, Siri won this battle, Cortana was voted "most likely to go to web search" and Google Now needs to go back to college and pick up a few Mandarin classes.

She's picking on Google Now for Mandarin because it dropped accuracy by more than half, which is bu hao.

Accuracy is important, because the better a virtual assistant can understand your query, the better it can return helpful results. Nothing will ever be perfect, because language is complex and things like names make it even more so, which is why it's also great that if Siri is pronouncing a name wrong, you can teach it the proper pronunciation. You can tap an error in transcription, edit it it, and Siri will return an updated result.

When you combine "Hey, Siri", which allows for optional voice-activation when plugged into AC power, the ability to post Facebook status updates, to ask how much time is left in a sports event, to play podcasts by show or episode, buy iTunes Radio songs or gift certificates, and search for tweets posted by specific people, it makes Siri even more useful to even more people for even more things.

Throw out a landmark like Statue of Liberty or Great Wall of China and you'll get maps or Wikipedia results giving you all the information you need.

Siri also does sequential inference, so if you ask about the capital of Germany, it will tell you Berlin. Then, if you ask what the population is, without mentioning a specific place, Siri assumes you're continuing the same conversation and gives you the population of Berlin. Or if you say "what's Ally's phone number", and then "call her", Siri knows to call Ally.

Apple does, intentionally, limit Siri's scope. As a company, Apple is against harvesting its customer data (opens in new tab). Apple doesn't want to be monitoring our web history, for example, or running analysis and transactions against our mail or calendars on its servers.

That means Siri is not allowed to do some of the things competing products that get access to far more of our information can do. Just liked security, privacy is continually at war with convenience and it's up to each company and each individual to decide how much of one they're willing to give up for the other.

Siri is also an interface for CarPlay, HomeKit, and Apple Watch, and personally, that's the kind of convenience I want.

Update: GlobalMe has published the testing procedure and result charts.

Rene Ritchie
Rene Ritchie
Contributor

Rene Ritchie is one of the most respected Apple analysts in the business, reaching a combined audience of over 40 million readers a month. His YouTube channel, Vector, has over 90 thousand subscribers and 14 million views and his podcasts, including Debug, have been downloaded over 20 million times. He also regularly co-hosts MacBreak Weekly for the TWiT network and co-hosted CES Live! and Talk Mobile. Based in Montreal, Rene is a former director of product marketing, web developer, and graphic designer. He's authored several books and appeared on numerous television and radio segments to discuss Apple and the technology industry. When not working, he likes to cook, grapple, and spend time with his friends and family.

76 Comments
  • Hey! What's with the image at the top of the article scrolling through a fixed window as you scroll the page down? That's Cool! How'd you do that?
  • Webpage or app? Sent from the iMore App
  • How'd you make the image scroll -- I'm using Safari on an iMac.
  • It's called parallax scrolling. It's just a javascript-powered graphical effect.
  • Sure, Siri might do language accuracy well, but across the boards, Google Now and even Cortana give you more convenience across the boards. A virtual assistant should be more than a voice that talks back to you. Google Now and Cortana make it to where all your important information is right there for you before you even need to ask. Siri is good, but when it comes to functionality, Google Now and Cortana take the lead. Sony Z3, Nexus 5, or Surface Pro 3
  • Sony, Nexus, Surface Pro. Yup, sounds anti Apple to me. Sent from the iMore App
  • Do I need to specifiy the past Apple products I've owned? No I don't. And rather than pointing out something that doesn't contribute to this conversation, how about make a valid point about Siri vs her competition? Sony Z3, Nexus 5, or Surface Pro 3
  • Google Now runs circles around Siri between my iPhone and Android... Posted via the Android iMore App!
  • He is right regardless
  • +1
  • Funny because I've used Siri, G Now, and Cortana and although Siri might be missing a couple features it is (at least for me) the far better Digital Assistant and the main reason has to do with this article. Google Now is a giant pain in the ass. I use Siri almost all the time when I am driving for text and Google Now is a huge mess. I can't even get a sentence out without one to two words being completely wrong. It isn't even close. Yes, it's often funny, but beyond that it is completely worthless. Siri, on the other hand, is almost always spot on and does what I want it do. I can't even tell you the last time I've used Google Now on my Nexus 5 or Moto X. I hear people saying that Google Now runs circles around Siri all the time and I just have to laugh as it is the same old story as the whole iOS vs Android. Sure, Android and Google Now have more features, but how often do people use them? For me, I will use the products that actually make my life easier and not get me frustrated every time I use them. This is the reason I not only continues to use iOS products, but also Siri is they just work and they work really well. I would rather have the three features I need day in and day out work 95% of the time than 100 features that work 40% of the time. Maybe thats just me.
  • You can always tell which articles are by Rene, just by the headlines. Rah Rah Rah, gooooooooooo Apple!!! smh
    #endofjournalism
  • This is an Apple site and I linked to Venture Beat. You can disagree with an article and argue its premise without being childish or rude. At least theoretically. :) Sent from the iMore App
  • Practice what you preach please. You have been condescending and rude when I ask valid questions about Apple vs competition and your points on switching to Apple just for Apple Pay. If Phil Nickinson can be cordial to iOS users on Android Central (hell, Phil likes Mac), then you could be just a little more respectful to people who use another OS that want to have open dialogue or learn more about iOS. Now I will agree with you, instances of people coming here and outright trolling is another story.
  • I can't claim to understand any of what you wrote above, but I'm happy to discuss this topic if you'd like to raise any particular discussion points?
  • You can't understand what he wrote??? Really?? And I'm not trolling from another site or platform. Almost everything I use is either iOS or OSX. Sent from the iMore App
  • For me the amount of fanboyism with his articles is astounding ..its like a apple press ... I m a general tech enthusiast who uses both apple n android products. .. In general I think android central and windows central are far better... There was a fair comparison between iPhone 6 plus and nexus 6 at android central which we can never get here.. Posted via the iMore
  • Psychopaths have trouble understanding what they don't want to understand. Posted via the Android iMore App!
  • you should note that this article was created by a company that competes with Google and counts Alibaba and Nuance (engine that powers Siri)as clients. It's a shame these things are hidden.
  • Apple is doing the localization here. Not sure much if any of it relies on Nuance, much less Alibaba? Sent from the iMore App
  • You're of the opinion Siri no longer uses Nuance for it's voice processing? Do you have any proof of this? I didn't intent to claim that Siri/Apple is using Alibaba for anything more that Alibaba is a major competitor to Google.
  • I believe I wrote "localization", which is what is key to the language accuracy. They were testing how well it understood, not converted.
  • correct, which still uses the Nuance engine as it understands multiple languages (56 so far). It's not like Siri only understands English and everything has to be translated before it can search.
  • The article/test was performed by one Sophie Salonga, a marketing assistant at GlobalMe. One of GlobalMe's featured clients is Nuance ( http://www.globalme.net/project/nuance ), which coincidentally is the speech processing engine behind Siri...and not Cortana or Google Now. GlobalMe itself appears built around multi-lingual services...which coincidentally is a main thrust of this test. I'm just a *weeeee* bit skeptical here.
  • Apple, I believe, does their own localizations. The Siri team isn't Nuance-dependant ;) Sent from the iMore App
  • Misleading at best. Apple's current team is moving away from Nuance, true, with a hiring binge in mid-to-late 2014: http://9to5mac.com/2014/06/30/why-is-apple-hiring-nuance-engineers-appar... after Nuance angered Apple in 2013 by publicly speaking about the engine: http://appleinsider.com/articles/13/05/30/nuance-confirms-its-technology...
    http://www.forbes.com/sites/rogerkay/2014/03/24/behind-apples-siri-lies-... So, unless you are saying Nuance has been completely ripped out and replaced primarily by engineers hired in the last 8 months, without any comment from Apple or the Apple investigative press, Siri is still dependent on Nuance. And, with Apple inarguably moving away from Nuance, this could be the last window for Globalme and Nuance to use Siri to advertise its own prowess to potential replacement customers-- all the more reason such things should be researched and disclosed, rather than simply linked.
  • That's a remarkable conspiracy theory. Since Apple is doing the localization, maybe it would be a chance for a "Nuance Shadow Cabal" to prove they can't do it without Nuance, and show accuracy levels in the decline? Maybe Paul Greengrass could film it as a Bourne flick? (I'm only being flippant to make a point :) )
  • What I read. Siri can understand you and replies with "Sorry I can't do that" in the language of your choice. The other options make you repeat yourself and do what you freaking asked it to do. SIRI IS BETTER! Posted from the amazing whatever device I can afford because I'm a broke college kid.
  • Lol Posted via the Android iMore App!
  • Hahaha. Love it Sent from the iMore App
  • Hahahaahaha spot on! Posted via the iMore App
  • Nice try, but no. My name is of non-English language origin (parents were hippies), and Siri can't even pronounce it right, despite my attempts at re-educating her countless times. Also, why does the English-UK Siri default to some pompous dude? Some lame attempt at being a butler? At least let me say "Hey Jeeves" if that were the case!!
  • Tried the same, and my name cannot be spoken properly even after trying to tell Siri how it should. To me the use of Siri is limited. I'm Italian, I live in Germany and I speak 4 languages that I use every day. Siri gets just once at the time, which means calls and messages are in most of the cases faster manually than using it. Sent from the iMore App
  • Google is clearly better in the the English speaking part of the world. Talk normal and fast and Google will amaze you. It amazes me on my nexus 6 every day.
  • Hey Rene, nobody believes you. Posted via the Android iMore App!
  • Belief is irrelevant. What did I write that's factually incorrect? Sent from the iMore App
  • Belief is indeed irrelevant, but the kernel of a complaint here is that imore has with great fanfare declined to link to articles that are clickbait, have conflicts of interests behind them, or are poorly researched, with the intention of denying traffic to that site. Nothing you wrote is factually incorrect, but it does appear that same standards were not applied to the venturebeat article, and the appearance of applying different standards to pro-Apple vs anti-Apple posts can be damaging to credibility long-term.
  • Are you suggesting the study was falsified?
  • Had an Android... had a Lumia... now giving this iPhone thing a try.
    It definitely "feels" like Siri is better at understanding stuff... but to be honest I never had issues with Cortana or Google Now in that area. I miss Cortana's personality and her ability to do stuff for me even without me asking explicitly.
  • Siri and Cortana having personalities are definitely one of their strengths. Google Now is more accurate with English, but has no personality and misses the mark on a few items to be a digital assistant. Sent from the iMore App
  • Google Now for me is never right and all I speak is English. Siri is far better at speech recognition in my humble opinion. It isn't even close. I wanted to use Google Now I really did, but lost my patience way to many times. What I need Siri does much better than anyone else. Google Now needs to get a lot better at speech recognition. You can have the few more features if Google Now doesn't understand what the heck you are saying in the first place.
  • I don't have an iPhone so I don't know why it would be worse than on my Nexus 6 but I find it works great. It's almost perfect. Now if you talk to it like we use to where you raise your voice and talk like a robot then NO it works like crap. If you talk at a normal volume and don't put your lips ups to the phone microphone and talk as if your talking to your friend it works almost perfectly. I'd love to show you.
  • Thanks for the reply. I have a Nexus 5. I am not saying Google Now doesn't ever work it just doesn't work well enough (for me) to use it daily. Sure, if I need to ask a question and it queries google.com and kicks back some nice looking card I always come away impressed, and it does it far better than Siri does. This is not how I use a digital assistant. On a regular day I use it to read my texts (if I receive one while driving), send a text (again, if I am driving), call my wife (which I always use), and maybe check a sports score, or set an alarm. Occasionally I will set an iCal event. Siri, Google Now, and Cortana do all of these and do them well, however, Apple's Siri in my opinion does it far better and gets the words right more often than Google Now or Cortana does. Google Now is often ridiculous in what it thinks I am saying. I live in Utah so I don't have an accent, talk normally (at least what I think is normal), and I don't talk fast. Like I mentioned in another post Siri seems to get what I am saying a good 90-95% of the time and the times it doesn't get it it is usually just a word off. Google Now on the other hand is the complete opposite. It only works a good 60-65% of the time. It is more frustrating than it is worth. I am hardly someone that uses digital asst a lot, but I am the kind of person that seems to not use something after it frustrates me and that is what Google Now is.
  • Moving away from the petty iOS vs Android snipes in the comments, the only part of the article I'd argue with is calling 76% remarkable, unless that because it's remarkable because it's so low. I realise that speech recognition is difficult but what these numbers are saying (if I've interpreted them correctly) that Siri gets every fourth word wrong and the competition gets at least every other word wrong. If I had a P.A. who transcribed dictation tapes that inaccurately they'd be out the door before Monday lunchtime!
  • Language really is hard. For me it's more about getting the gist of what I'm dictating so I don't forget it. I can clean it up with type later.
  • While I like Siri, I don't think it's heads over heels compared to Google Now. Siri has its good spots and bad spots. Posted via the iMore App for Android
  • I can use Siri without having to hand over gobs of personal data, which makes a huge difference to me. Google Now literally wouldn't let me turn it on when I said it couldn't access my web history. (Maybe that's since changed?)
  • I didn't realize you can turn off Siri personal data collection. Is Siri any worse when it doesn't collect gobs of personal data?
  • Siri, for example, doesn't care about my web history. Apple, for example, doesn't care about digging through my data in their server space. So yeah, it's way better about personal data.
  • "Siri, for example, doesn't care about my web history."
    Wow. I didn't realize Apple allows iOS apps to curate through users web history. That's a nasty privacy feature, thanks for bringing it up. "Apple, for example, doesn't care about digging through my data in their server space." - According to the Apple privacy policy, Apple does dig through your data to improve Siri accuracy, why on earth do you think they keep your data on their servers? It reads like you aren't aware of what Apple does with your data.
  • You can argue it however you want, but it comes down to one simple fact. Apple is a hardware/software company that sells directly to their customers and no one in between. Google on the other hand is primarily (like 95% of all their revenue) ad selling company. They make their cash by selling your data so people can force feed you ads. As for me I will give my money over to Apple as at least I am a direct customer. Google your not even the primary customer their ad partners are. Who do you think can be trusted more with your data? I love how Android fans act like this isn't fact. Rene is right. The data Google has on all of us is frightening to say the least. How they make themselves a good guy is beyond me. The only thing more surprising than that is how all you Android fans stick up for them like they do no wrong because they bought Android and a lot of talent to make it a competitor. Just because I like Android doesn't mean I'm willing to hand my data over and then turn around and stick up for them like my life depends on it.
  • Google doesn't give your data to anyone. They have it. They keep. They don't sell your actual information to anyone. What they do is have a ton of different ads and place the ad with the customer who they think is most likely to be looking for that product. Sure, they could know everything about you. They can strip your person info out and just sell general stuff like 18 to 25 year males mainly search for ...... But I'm tired of acting like Google goes out and says $15 for Tim Cooks data to anyone who wants it. They keep all my data locked up and the second I want to delete it I can. http://www.google.com/intl/en/policies/privacy/#infochoices Either way Google isn't a charity. They have to pay for everything they make. I use Google everything and it would be fair to charge me $20 to $100 a month for everything I use of theirs. That model doesn't work so they instead went with the free tv model of trading ads for services. Difference is it in't 1970. Ads are smartly targeted to get the most bang for the buck which is why Google gets as much money as it does from ads.
    If Rene doesn't want to give Google any of information that fine. If you don't then that is fine as well. Not everyone has money to pay for everything. I for one rather get Google's wealth of services for me giving my data. There is so much value I get out of it and repeating everything that Google Now can do with it on my Android phone isn't going to matter to some but unless you've experienced it you have no idea. The richest 80 people in the world have 99% of the worlds wealth. http://www.ibtimes.com/richest-1-will-have-more-wealth-remaining-99-2016...
    Google's business model isn't Apple's. They want everyone to use their software not the just the ones clearing $60,000 + plus a year. Why is that such a bad thing? Google's doing all they can to keep all their systems safe.. I haven't seen them in the news recently with any hackers stealing my G+ photos or anything of the sort.
  • This is exactly what my post was about. You can agree or disagree with what Google does as a company, but you're looking past something that I no longer will. You can try and downplay what Google does with your data, but what they have access to and what their partners have access to is a virtual treasure trove of information about what you do while on the internet, where you drive your car, what restaurants you go to, what you buy online, what pictures you take and share, your email and it's contents, and now with Android almost everything you do on your phone and/or tablet (if there even is people that buy Android tablets now a days). Google is not the good guy company you think they are. Google makes 96% of all their profits on two main parts of their business. Adwords and Adsense. Adwords they make off of people clicking on ads and Adsense sells their trove of information to partners to sell you ads. That is all of Google's income. You are nothing more than a customer to their customers. For example, I've always used gmail for years and have felt the same way you have where I have nothing to hide and as long as Google keeps making great free software they can use my info. I was signed into my gmail account and my brother was going through a hard time and staying at my house. This was three years ago and he happened to search for divorce lawyers. Since then I can't tell you how many ads I've gotten about divorce attorneys, therapists, books, online help, and even suicide hotlines. I am willing to bet I've seen literally 300 ads all from this one search my brother did over three years ago. So if you want to keep kidding yourself that they only sell certain info you are wrong. When it comes to making money Google will sell whatever it has to curate ads whether that be your search queries, email, google maps data, and even the apps you use on Android. They do this and often act like it is a joke as their mantra is don't be evil. What corporation in the world would we deal with having a mantra that has to reming them everyday to not be evil. This goes deeper than most probably care to understand. You can choose to let them do what they will with your data, but as for me it simply comes down to buying into what Apple is selling where I am a customer and have great customer services with products that aren't for sale. I can get my new iPhone or Apple product and know it isn't plugged full of garbage bloat ware. I can feel like what I am doing online is not being sold to others in order to sell me ads. They are completely different ways of doing things and at the end of the day it simply comes down to corporations are around to please shareholders and make money. Apple is out to make great products and services while making money. Google on the other hand is out to make software to get as much data on people as they can as the more they have the more they can sell to their real customers.
  • Ok, So Google correctly targeted ads at you because your brother was logged in as you looking for "divorce lawyers" They targeted ads that relate to this. Why is that a problem? .It works just like it should. Why a