DeepMind and YouTube release Lyria, a gen-AI model for music, and Dream Track to create AI tunes

In January, Google made some waves — soundwaves, that is — when it quietly released some research into AI-based music-making software that builds tunes based on word prompts. Now, its sister company Google DeepMind has gone a step further: it has Office has partnered a new generation model of music called Lyria that works partner with YouTube; and two new toolsets it describes as “experiments” built on Lyria. Dream Track will allow creating music for YouTube Shorts; and Music AI tools that are said to be aimed at helping the creative process: for example creating a tune from a snipped that a creator can hum. Along with this, DeepMind said it adapted SynthID – used to mark AI images – to watermark AI music as well.

The new tools were released at a time when AI continues to court controversy in the creative arts world. It’s a key topic at the heart of the Screen Actors Guild strike (which finally ended this month); and in music, while everyone knows that Ghostwriter used AI to imitate Drake and The Weeknd: the question you have to ask is whether AI creation will become more common in the future.

With the new tools announced today, the first priority for DeepMind and YouTube appears to be creating technology that helps AI music remain reliable, both as a complement to current creators, but also in the most aesthetic sense of sound like music.

As shown in Google’s previous efforts, a detail that often emerges is that the longer one listens to AI-generated music, the more distorted and surreal it starts to sound, deviating from the desired result. . As DeepMind explained today, that’s partly due to the complexity of the information that goes into music models, which include beats, notes, harmonies and more.

“When creating long sequences of sound, it’s difficult for AI models to maintain musical continuity in phrases, verses, or extended sentences,” DeepMind said today. “Because music always has many voices and instruments at the same time, it is more difficult to create than to speak.”

It is noted, then, that some of the early applications of the model featured shorter pieces.

Dream Track initially launched with a limited set of creators to build a 30-second AI-generated soundtrack of “the voice and musical style of artists including Alec Benjamin, Charlie Puth, Charli XCX, Demi Lovato, John Legend, Sia, T-Pain, Troye Sivan, and Papoose.”

The creator enters a topic, chooses an artist, and a track with lyrics, backing track, and the voice of the chosen musician is used to create the 30-second piece, which is intended to be used with Shorts . An example of a Charlie Puth track here:

YouTube and DeepMind clearly point out that these artists are involved in the project, helping to test the models and providing other inputs.

Lyor Cohen and Toni Reed, respectively head of music for YouTube and its VP of evolving experiences and community projects, note that the suite of Music AI tools released from The company’s Music AI Incubator, a group of artists, songwriters. and producers who work to test and provide feedback on projects.

“It was clear early on that this first group of participants was extremely curious about AI tools that could push the limits of what they thought was possible,” they said. “They’re also looking for tools that enhance their creative process.”

While Dream Track is getting a limited release now, the Music AI tools will only be released later this year, they said. DeepMind said there are three areas they will cover: creating music with a specific instrument, or creating an entire set of instrumentation, based on humming a tune; use the chords you build on a simple MIDI keyboard to create an entire chorus or other ensemble; and build backing and instrumental tracks for vocal lines you may already have. (Or, actually, a combination that uses all three, starting with a simple hum.)

In music, Google and Ghostwriter, of course, are not alone. Among other tool launches, Meta open sourced an AI music generator in June; Stability AI launched one in September; and startups like Riffusion are also raising money for their efforts in the genre. The music industry is also trying to prepare.

Leave a comment