Games and Music Receive Top Amazon Alexa Skill Reviews
Yesterday we revealed that 62% of Amazon Alexa skills have no user reviews and 5% have 1-star ratings. Those one-star ratings are the worst you can give to a skill. To build on those findings, Voicebot analyzed the 100 Alexa skills with the most user reviews. We found that skills in the “Smart Home” category showed up most frequently in the top 100 list. These were followed by skills in categories “Games, Trivia & Accessories,” “Music & Audio,” “News,” “Food & Drink,” and “Lifestyle.”
The presence of these categories in the list of skills with the most reviews may tell us more about what users are motivated to leave a review than which skills are most popular. We know from other market data that the most used skills fall into the “Music & Audio” category and yet it is third in total entires in the top 100 most reviewed skills.
Interestingly, this popularity shows up in the ratings. It is important to make the distinction between total number of reviews and user ratings for skills. The second most reviewed skill, TP-Link Kasa, only has a average of 2.5-star rating. By contrast, the Free Sleep Sounds skill has about one-tenth the total number of reviews but a 4.7-star rating. It falls in the “Music & Audio” category which outperforms “Smart Home” category skills by an average of 46%. The “Music & Audio” skill with the most reviews is SiriusXM. It has the third most reviews at 1,240 and an average rating of 3.0-stars.
Music & Games Reflect the Highest Rated Skills
When you analyze just the 100 skills with the most reviews you find that skills in the “Music & Audio” and “Games, Trivia & Accessories” categories outshine others by a meaningful margin. The average rating for “Music & Audio” skills from the top 100 most reviewed list is 4.38-stars. For “Games, Trivia & Accessories” skills in that list it is 4.13-stars. The average user rating across 17 skill categories is 3.18-stars and the average for all skills in the 100 most reviewed is 3.41-stars.
The worst performing categories are “Productivity” and “Local” followed by “Connected Car” and “Weather.” All of these categories represented less than 3% of total skills in the most reviewed list.
How Reliable Are Alexa Skill Ratings
Several developers reached out to me yesterday to express concerns about how reviews are made and ratings given. There is a sense that some skill developers are gaming the system to drive up either total number of reviews or ratings. But, everyone is playing by the same rules so this is really a matter of choice in terms of how you build, support and promote your skills.
More convincing was the argument that earlier problems caused by Amazon outages led to a number of skills getting several low ratings by users. Newer skills weren’t around for those platform issues and therefore didn’t receive the perceived ratings penalty. That may be true, but it is not clear what you would do about it. One option would be for Amazon to suspend reviews in a period when technical issues on the platform are negatively impacting skill performance. This is a possibility but would be difficult to govern and is not likely to be near the top of Amazon’s priority list.
The key lesson of this analysis is that total number of reviews and ratings are just one signal users should take into account when considering whether to try out a new skill. However, the fact that so few users ever access the Alexa app and actually look at skill ratings suggests it will not have a big impact on how many people use a skill or how they feel about it. Media coverage and positive word-of-mouth recommendations are far more important today in inducing trial. The quality of the skill concept, design and performance are the real drivers of user satisfaction and retention.
Another Entry Into the 1,000 Alexa Skill Reviews Club
Yesterday, we revealed that there were only four Alexa skills with more than 1,000 user reviews. Invoked Apps founder Nick Schwab informed me that his skill, Sleep Sounds: Thunderstorm Sounds, joined that club this morning and currently stands at 1,030 reviews with a 4.3-star average rating. So, now we have five Alexa skills in the 1,000 review club. Congratulations Nick.
62 Percent of Alexa Skills Have No Ratings, But 4 Have Over 1,000
