DeepMind Claims New AlphaCode AI Programmer Outperforms GPT-3 and is Competitive With Humans
Google-backed DeepMind has unveiled a new computer programming AI called AlphaCode that it claims performs better than OpenAI’s GPT-3-powered Codex tool. AlphaCode scored a significant win at a recent Codeforces programming contest, ranking on average in the top 54% against more than 5,000 participants in 10 contests.
AlphaCode AI
AlphaCode, a reference to Google parent company Alphabet, is the first to reach its competitive level. The program understands plain language instructions into computer languages. The output is usually accurate and ready to plug into existing operating systems. There’s a lot of potential for AI-powered code. Anything that reduces the time and resources spent by human developers will reduce overall costs and give coders space to work on the more creative elements of program design.
AlphaCode is trained on public GitHub databases and can produce useful computer code in nearly a dozen programming languages from natural language descriptions. The 715.1 gigabytes of data collected for the AI made up a gargantuan 41.4 billion parameters. OpenAI’s Codex is about a quarter that size despite processing around 600 gigabytes of data. Codex uses a version of the GPT-3 language model so there would be no need for programming training to build software. The comparison is one that DeepMind makes in their academic paper on AlphaCode, though mainly to showcase how it outperforms the GPT-3 version.
“The most relevant work to ours is the recent Codex system, a GPT language model trained on public code from GitHub. This model demonstrated impressive performance, achieving a high success rate at correctly completing hand-specified Python functions given the function signature and docstring, especially after fine-tuning on a similar dataset,” DeepMind’s researchers explained. “However, the programming tasks these works address are simple compared to the full scope of competitive programming problems, where both the task specification and the solutions are more involved. Our work uses transformers but pushes model performance a significant step forward, from generating function completions to creating full solutions to held-out competitive programming problems.”
Codex Competition
AlphaCode is still very new, so it may be a while before it becomes widely available. OpenAI is ahead in that regard, with a Codex API available for developers. Codex is also part of the Github Copilot introduced last year. Copilot uses the AI as a “pair programmer,” mimicking when two developers simultaneously work on a coding project and share comments as they work. The AI is the junior partner and used Codex to understand what the programmer is doing and come up with suggestions. Copilot is supposed to be able to suggest solutions to problems, come up with tests, and even brainstorm features while tailoring itself to the user’s working methods. OpenAI also debuted a new default model called InstructGPT to better align with user intent and cut down on offensive or just nonsensical responses to requests.
Follow @voicebotai Follow @erichschwartz
OpenAI Codex Running Beta for Natural Language to Computer Code API
OpenAI and GitHub Unveil New Copilot AI Assistant for Coding
