Meta Releases Open-Source Generative AI Text-to-Music Composer MusicGen
Meta has released a generative AI model for producing music named MusicGen, capable of turning a written prompt and music sample into an original composition. MusicGen is open source, at least to a limited extent, but offers developers much more room for experiments compared to Google’s MusicLM generative AI music composer, which has been seen only in a few demonstrations.
MusicGen
MusicGen can perform music based on a written description, though the samples are only about 12 seconds long. Users can supplement the writing with a short audio clip as a reference for the AI to build on. Meta trained MusicGen on 20,000 hours of music, about half of which the researchers said is licensed and high-quality music tracks, as well as 390,000 instrumental songs curated from stock media libraries Pond5 and Shutterstock. MusicGen is open-source, but Meta isn’t making the training code available yet. Interested developers can play with available pre-trained models if they have the right hardware.
MusicGen and MusicLM are not the only musical generative AI model around. For instance, the visual and sonic AI project Riffusion uses Stable Diffusion to turn a text prompt into a sonogram, a visual representation of sound as a graph with time on the horizontal and sound frequency on the vertical axis. Riffusion then uses Torchaudio to read the frequency and time to play the sound. There are also AI tools for blending generative AI and music, like Voicemod’s synthetic song generator, which matches submitted lyrics to a selection of popular songs and AI voices, and the text-centered LyricStudio, which claims its AI has assisted in writing more than a million songs.
Sing AI
MusicGen and its compatriots aren’t putting composers on the streets yet. The legal and ethical issues, not to mention technical limits, can’t compare. That said, the space is evolving at an extraordinary pace. AI-created songs, especially with deepfake stars performing, can draw millions of listens. Musical artists like Grimes, Timbaland, and David Guetta are keen to experiment with the technology. Holly Herndon outright offers a free synthetic version of her voice to make music with called Holly+, and the musical group YACHT trained an AI model to write an entire album called “Chain Tripping.” Still, labels like Universal Music Group are asking Spotify and other platforms to prevent AI developers from using the music it owns for training AI, and streaming service Deezer is even working on building AI tools to detect and remove deepfake singers and synthetically generated songs from its platform.
Follow @voicebotaiFollow @erichschwartz.
Music Streaming Service Deezer Building AI to Hunt and Remove Deepfake Songs
