Stability AI, the company known for the AI-driven art creator Stable Diffusion, has introduced a new open-source AI model for generating sounds and music. This model, named Stable Audio Open, claims to be trained solely on royalty-free recordings.
Stable Audio Open creates audio based on text prompts (e.g., “Rock beat played in a treated studio, session drumming on an acoustic kit”), producing recordings up to 47 seconds long. The model was developed using approximately 486,000 samples from free music databases like FreeSound and the Free Music Archive.
According to Stability AI, the model is versatile and can generate drum beats, instrument riffs, ambient sounds, and other “production elements” suitable for videos, films, and TV shows. It can also modify existing songs or apply the style of one genre (e.g., smooth jazz) to another piece.
“One major advantage of this open-source model is that users can fine-tune it with their own audio data,” Stability AI stated in a blog post. “For instance, a drummer could customize it with their own drum samples to create new beats.”
Despite its capabilities, Stable Audio Open has limitations. It cannot generate complete songs, melodies, or vocals to a high standard. Stability AI notes that the model is not optimized for these tasks and recommends their premium Stable Audio service for such needs.
Additionally, Stable Audio Open is restricted from commercial use, as per its terms of service. It also shows varying performance across different musical styles and languages other than English, reflecting biases from the training data.
“The data source may lack diversity, leading to underrepresentation of various cultures,” Stability AI explained. “Consequently, the model’s outputs will exhibit these biases.”
Stability AI has faced challenges, including a recent controversy following the resignation of its VP of generative audio, Ed Newton-Rex, who disagreed with the company’s view that using copyrighted works for training AI models is “fair use.” Stable Audio Open seems to be an effort to shift this narrative and promote Stability AI’s premium offerings.
As AI music generators like Stability AI’s grow in popularity, the issue of copyright abuse has come to the forefront. In May, Sony Music, representing artists such as Billy Joel, Doja Cat, and Lil Nas X, issued warnings to 700 AI firms against the unauthorized use of its content for training audio generators. Additionally, in March, Tennessee enacted the first U.S. law aimed at curbing AI abuses in music.
News of AI Google