Stanford Closes Meta LLaMA-Based Alpaca Generative AI Demo Over Safety and Cost Problems
Stanford University has ended the web demo of its Alpaca generative AI model after only a few days after concerns about safety and cost made it untenable to continue. The AI model is based on Meta’s LLaMA (hence the name), which has a history of problems with its generative AI chatbots shutting down soon after their release.
Alpaca Packed Up
Meta created LLaMA with a plan to encourage LLM research, particularly issues of wrong or inappropriate text generation. LLaMA was designed so that those researchers wouldn’t need expensive hardware to begin their work. There are four different LLaMA models, and Meta claimed that LLaMA-13B and its 13 billion parameters could outperform OpenAI’s GPT-3 in most metrics and only required a single fo the (still high-end) Nvidia Tesla V100 GPUs. Stanford said Alpaca costs just $600 to build and fine-tune from the smaller LLaMA model, the LLaMa-7B. That was a deliberate choice to prove the relatively low cost of replicating what OpenAI and its rivals are doing.
“Many users now interact with these models regularly and even use them for work. However, despite their widespread deployment, instruction-following models still have many deficiencies: they can generate false information, propagate social stereotypes, and produce toxic language,” the researchers explained at the time. “We train the Alpaca model on 52K instruction-following demonstrations generated in the style of self-instruct using text-davinci-003. On the self-instruct evaluation set, Alpaca shows many behaviors similar to OpenAI’s text-davinci-003, but is also surprisingly small and easy/cheap to reproduce.”
But while it was inexpensive to build, Alpaca was apparently not cheap to host. And while Alpaca was built partly to aid in identifying and cataloging toxic text and other issues with LLaMA, the researchers decided it would be best to end the demo. That’s even after launching the demo with an acknowledgment of Alpaca’s penchant for hallucinating an array of wrong answers like the capital of Tanzania or being willing to write misinformation with no resistance.
“The original goal of releasing a demo was to disseminate our research in an accessible way,” Stanford said in a statement. “We feel that we have mostly achieved this goal, and given the hosting costs and the inadequacies of our content filters, we decided to bring down the demo.”
The decision is reminiscent of when Meta last tried to share a demo of a generative AI project. The Galactica Large Language Model demonstration lasted just three days back in November. Designed to show how generative AI could write academic papers, Galactica quickly attracted a ton of online ridicule over nonsensical and problematic essays. Meta was quick to remove access to it as a result. Sharing LLaMA only with academic researchers might have been partly to avoid that issue, but clearly, any research on generative AI with a public face could be a problem. That might be why LLaMA didn’t include much, even in the way of approved demonstrations, compared to when Meta showcased its Make-A-Video synthetic media engine.
Follow @voicebotaiFollow @erichschwartz
Meta Introduces Large Language Model LLaMA as a Competitor for OpenAI
