Meta keeps sharing its artificial intelligence developments. In mid-April we explored Animated Drawings to animate drawings, its chatbot Llama 2 appeared in the last days of July, and now it's AudioCraft's turn. As its name suggests, AudioCraft focuses on audio generation with artificial intelligence, and consists of three models: AudioGen, MusicGen, and EnCodec. The code is already available for everyone, but if you want to try MusicGen right now, you can do it without installing anything.
Riffusion, MusicLM, Soundraw, SongR. The idea of generating music with artificial intelligence is gaining traction on the Web. From a technical point of view, those platforms are still far from image generation, but time is definitely on their side, and with each new model, the quality parameters will continue to improve.
That brings us to the AudioCraft project from Meta. This library was specially designed for deep learning research in audio generation, and is divided into three advanced models: AudioGen for creating sound effects, MusicGen for generating music, and EnCodec, a high-fidelity "neural audio codec". The AudioCraft profile on GitHub contains all the code and installation instructions, but you can try MusicGen right now thanks to Meta's presence on HuggingFace.
How to generate music with prompts using AudioCraft and MusicGen
Two tools are available in this version of MusicGen: a box to enter our prompt, and another to load a melody or microphone sample as a reference. We click Generate, and that takes us to the waiting queue. On average, HuggingFace reports a delay of 600-900 seconds, but in some tests, the model took half that time (of course, this will depend on server load). The final result is a fifteen-second sample in MP4 format.
"a lively bard playing the lute on a tavern"
"an ominous, foreboding church piano melody, anticipating the appearance of an enemy in a videogame"
"soundtrack of a ecstatic moment in a space battle of a sci fi movie"
How well does it work? Well... it's not terrible. The model doesn't offer any seed or intensity control, but it seems to follow our prompt with some accuracy. Obviously, the description is written in English, and any minimal modification can generate very different melodies. The fifteen-second limit is absolute here, and we shouldn't expect a loop for background use, but I think MusicGen has the potential to become a much more robust tool.
Access MusicGen: Click here. AudioCraft on Facebook: Click here.
