This review discusses the interim results of a blind listening test involving piano loops generated by eight different large language models (LLMs). The study aimed to determine which model's compositions were preferred by listeners, providing insights into the capabilities of these AI systems in the realm of music composition.
Who is it for?
This study is particularly relevant for musicians, composers, AI researchers, and enthusiasts interested in the intersection of artificial intelligence and music. It provides valuable insights into how various LLMs perform in generating musical compositions, which could influence future developments in music technology and AI-assisted creativity.
✅ Pros
- Provides empirical data on listener preferences for AI-generated music.
- Highlights the strengths and weaknesses of different LLMs in music composition.
- Offers a structured approach to evaluating AI-generated content through blind testing.
❌ Cons
- Results are interim and may change as more data is collected.
- Limited to one generation per prompt, which may not capture the full potential of each model.
- Participants were untrained listeners, which may affect the reliability of the results.
Key Features
The study involved 62 listeners who made 514 pairwise judgments on eight-bar piano loops generated by eight LLMs: Gemini, Claude, Qwen, Kimi, Grok, GPT-6.1, DeepSeek, and Llama. Each model was tasked with creating compositions based on the same 20 prompts, resulting in a structured comparison of their outputs. The loops were rendered using a consistent piano sound, ensuring that differences in listener preferences could be attributed to the compositions themselves rather than variations in sound quality.
Pricing and Plans
As this review focuses on a research study rather than a commercial product, there are no pricing or subscription plans to discuss. However, the findings may have implications for future tools and services that leverage AI in music composition.
Alternatives
While this study specifically evaluates the performance of eight LLMs, there are numerous other AI tools and platforms available for music composition. Some notable alternatives include OpenAI's MuseNet, AIVA, and Amper Music, each offering unique features and capabilities in generating music.
Best For / Not For
This study is best for those interested in understanding how AI can contribute to music creation, particularly in a comparative context. It may not be suitable for those seeking definitive conclusions, as the results are still interim and subject to change as additional data is collected and analyzed.
The interim results of this blind listening test provide intriguing insights into listener preferences for AI-generated music. With Gemini 4 Argon emerging as the most preferred model, the study highlights the potential of LLMs in music composition while acknowledging the limitations of the current findings. As more data is gathered, the conclusions may evolve, making this an area worth monitoring for future developments in AI and music.