๐ LLM
AI generated
CSyMR: Benchmarking Compositional Symbolic Music Reasoning With LLMs
## CSyMR: Evaluating Compositional Symbolic Music Reasoning with LLMs
Large Language Models (LLMs) are increasingly leveraged in symbolic music reasoning. However, existing benchmarks often emphasize isolated knowledge or atomic analyses, rather than the integrative compositional reasoning needed to connect musical structures.
To address this limitation, the Compositional Symbolic Music Reasoning Benchmark (CSyMR-Bench) has been presented, a curated multiple-choice dataset derived from expert forums and professional examinations. Each item involves combining several atomic analyses to arrive at the final answer.
Furthermore, a tool-augmented agent framework has been introduced that leverages symbolic music analysis tools from the music21 library to address the challenges posed by CSyMR-Bench. Experiments have validated that CSyMR-Bench poses a non-trivial challenge across both community-sourced and exam-style questions, while the tool-augmented agent consistently outperforms all baselines, achieving 5-7% absolute accuracy gains.
This work represents a significant step forward in evaluating and improving the musical reasoning capabilities of LLMs, opening new avenues for the integration of music analysis tools in the reasoning process.
๐ฌ Commenti (0)
๐ Accedi o registrati per commentare gli articoli.
Nessun commento ancora. Sii il primo a commentare!