Thousands of Google Actions Mysteriously Disappeared from Online Directory and Are No Longer Available, Google Says it Could Take Weeks to Resolve
Editor’s Note: This is an evolving story. The topics in this article are still highly relevant and you can also read about some new information on the extent of the impact by country here.
For the past two days, there has been some confusion among a number of Google Action developers because their voice apps mysteriously disappeared from the Google Actions online directory and they have not been accessible through Google Assistant. At about 8:00 pm CST on Wednesday, October 16th, a post in the GoogleAssistantDev subreddit went up saying “multiple Actions are not functioning.”
This was soon followed by another post from a developer reporting that he or she had received a customer complaint earlier in the day around 11:00 am CST saying an Action was not available. Other developers confirmed they were experiencing the same issue with one going further to say, “Mine is working from the simulator, but it’s not working from any actual device.” The Actions that were no longer callable by Google Assistant also had directory listings removed and replaced with an error message saying that “We can’t find what you are looking for at the moment.”
A similar set of posts appeared in the Google Home developer Slack channel around 6:00 pm CST Wednesday. Jo Jacquinta said he had received an “unhealthy” action notification earlier in the day and then an “it’s healthy now” notification. His popular 6 Swords game appears to be affected as are others.
Finally, around 3:20 pm CST on Thursday, October 17th, a post was made in the GoogleAssistantDev subreddit from taycaldwell who is listed as a Googler and appears to have made no other posts on Reddit. The post states that there is a known issue and the Actions on Google team was “conducting a comprehensive review to ensure that they meet our developer policies.”
More Likely a Bug Than and Policy Issue
A Voicebot inquiry from earlier in the day to the Google PR team eventually pointed me to the taycaldwell subreddit post implying it was the official statement from the company. The post is perplexing because it appears to suggest the developers may be at fault for violating “policies.” These seems unlikely for a number of reasons. First we can look at the types of Actions that are down. Stuart Crane from VoiceMetrics first notified Voicebot about the issue saying:
Seems like a pretty big outage – we have dozens of SurveyLine actions that are all no longer working, and it’s easy to find hundreds of other Google Actions that are down.
Crane went on to say “Expedia, Audioburst, ABC News Update, Tide, Quaker, Levi’s, Wells Fargo, Hoover Dry, IBM Watson, AT&T Internet, and a ton of others – those are just a few that we have found that are no longer working. There are many more.” Sadly, Voicebot Says is among the casualties and we just started posting again. And, CBS Sports’ Google Action still shows up as a Google search result but a click through to that link shows a similar error page to the one listed above. Is it likely that this many Google Actions from these high profile organizations all had suddenly run afoul with Google Assistant policies?
In addition, a Googler was kind enough to look at their internal dashboards for me. When reviewing several Google Action titles there was no indication of errors, but a couple no longer showed up at all. One thing that did show up was a notification about an increase in support tickets. We can surmise those were related to the unexplained Action outage.
Thousands of Google Actions Impacted
Google goes out of its way to make it difficult to identify how many Actions are listed in its online directory by constantly repeating Actions across categories and subcategories. However, Voicebot did a comprehensive analysis in January 2019 that included deduplication and removal of ghost Actions that were really just scrapes of webpages. However, we can get a sense of the scale of the current problem even without going through the deduplication process by simply looking at the gross number of Actions that were previously listed in each subcategory and comparing those to the January 2019 figures.
An analysis today of the Arts & Lifestyle, Business & Finance, Communications & Social, Education & Reference, Food & Drink, Games & Fun, and Health & Fitness categories showed that thousands of Actions are no longer listed in subdirectories where they were present just a few months ago. Some of these missing Actions were duplicated across multiple subcategories but many are not and our analysis represents fewer than half of the Google Actions online directory categories. This is a widespread issue. There are thousands of Actions missing.
The Google Action directory is also exhibiting other unusual behavior. The main category listings pages show one to three entries for a subcategory and then you can click through to “view more” Actions. Some of these subcategories listings used to have a “view more” option, but no longer do and only show the 1-3 featured listings. Featured Actions from subcategories also used to be listed as well when you click through to the page behind the “view more” links. Now, these are sometimes shown, but other times one or two are shown but not another that was featured at the category level. There does not appear to be a pattern to this which may make the source of the bug harder to find.
Does Not Appear to Be Dialogflow Driven and Could Take Weeks to Resolve
Dialogflow 1.0 is due to be deprecated later this month so that would seem to be a potential source of the problem. And, there was a global Dialogflow outage which took out all Google Actions earlier this year for several hours. However, Voicebot reached out to a number of developers that use Dialogflow and their Actions were all live while other developers using the service were down. So, Dialogflow seems to be an unlikely source of the problem.
Google did roll out a number of new Assistant features on 15th alongside its hardware preview event so some developers have suggested there could have been some bugs associated with those. However, those updates around smart home would seem unlikely to impact the Actions that are currently affected. Google appears to have little insight into the source of the problem today. In addition to the internal dashboards not indicating problems that were clearly taking place in the field an email to some developers about their support ticket included some additional information beyond the subreddit post saying, “Note this process may take up to a few weeks.”
It is worth noting that no embedded capabilities of Google Assistant, what they internally refer to as “Zero P” or “0P”, appear to be impacted. For those of you keeping track 0P in Google Assistant vernacular is equivalent to 1P in the Alexa skill universe. These are the native capabilities completely under the control of the Assistant team. The outage appears limited to a subset of 3P, known as third-party, Actions.
It is a fair question to ask whether Google would be communicating that this might take “a few weeks” if its core 0P Actions were impacted. In addition to independent developers, many large corporations that Google actively recruited to support the Assistant platform with Actions are down. You may wonder how they are feeling about their investment when their voice apps are not available and Google’s response seems to lack both urgency and transparency. Let me know what you think on Twitter.