What you need to know

  • Loup Ventures performed tests and found that Siri continues to improve when it comes to understanding and answering questions.
  • Although Siri has improved the assistant still lags behind Google Assistant.
  • The data shows Siri is particularly good at "phone-related functions."

A new study from Loup Ventures finds that when it comes to understanding and answering questions, Apple's Siri has gotten smarter with each passing iteration. In fact, the data shows that Siri is closing the gap on Google Assistant.

The study asked today's top three smart assistants—Siri, Alexa, and Google Assistant—the same 800 questions covering five categories. Some of the questions include: "Where is the nearest coffee shop?," "Who do the Twins play tonight?," and "How do I get too Uptown on the bus?" The questions were designed to test the acumen of these assistants during everyday scenarios.

It's worth noting, however, that the smart assistants were quizzed on smartphones, not smart speakers, where Assistant and Alexa primarily call home. Loup Ventures said it did this because "use cases vary," and the underlying technology is similar.

For instance, the environment in which they are used may call for different language, and the output may change based on the form factor; e.g., screen or no screen. We account for this by adjusting the question set to reflect generally shorter queries and the presence of a screen which allows the assistant to present some information that is not verbalized.

Although Siri was unable to best Google Assistant, Apple's technology did outperform its rivals in some situations and is slowly closing the gap on Google.

According to Loup Ventures, Siri is particularly adept at phone-related functions, such as calling, texting, emailing, calendar, and music.

Ultimately, Google Assistant came out on top, correctly understanding all 800 questions and answering with 93% accuracy. Siri, meanwhile, was able to understand nearly all 800 questions and answered with 83.1% accuracy. According to Loup Ventures, Siri falls short when answering questions related to commerce and information.

You can get more insight into the test from Loup Ventures right here.